Job failed/working slow

[#1]

If Hadoop jobs are working slow what are the process that we follow the troubleshooting.

[#2 · Accepted solution]

@Mohammad Shamim

The troubleshooting depends on which kind of Job are you running.

However if a job is running slow then broadly there may be any of the following reason which needs to be check as First Ad.

1. Please check if there are any slow responding threads or threads that are consuming more CPU cycles?

In such scenarios it is better to collect Thread dumps and the CPU metrics to understand the thread level issues/slowness/hung/CPU utilization etc.

https://community.hortonworks.com/articles/72319/how-to-collect-threaddump-using-jcmd-and-analyse-i....

.

2. Please check the memory utilization by your Job and to find out if the memory that is allocated for your job execution is sufficient or needs some tuning. In that case based on the nature of your Job please check the following links:

https://community.hortonworks.com/articles/22419/hive-on-tez-performance-tuning-determining-reducer....
https://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_spark-component-guide/content/ch_tuning-...

.