Support Questions
Find answers, ask questions, and share your expertise

Job failed/working slow

If Hadoop jobs are working slow what are the process that we follow the troubleshooting.

1 ACCEPTED SOLUTION

Super Mentor

@Mohammad Shamim

The troubleshooting depends on which kind of Job are you running.

However if a job is running slow then broadly there may be any of the following reason which needs to be check as First Ad.

1. Please check if there are any slow responding threads or threads that are consuming more CPU cycles?

In such scenarios it is better to collect Thread dumps and the CPU metrics to understand the thread level issues/slowness/hung/CPU utilization etc.

https://community.hortonworks.com/articles/72319/how-to-collect-threaddump-using-jcmd-and-analyse-i....

.

2. Please check the memory utilization by your Job and to find out if the memory that is allocated for your job execution is sufficient or needs some tuning. In that case based on the nature of your Job please check the following links:

https://community.hortonworks.com/articles/22419/hive-on-tez-performance-tuning-determining-reducer....
https://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_spark-component-guide/content/ch_tuning-...

.

View solution in original post

1 REPLY 1

Super Mentor

@Mohammad Shamim

The troubleshooting depends on which kind of Job are you running.

However if a job is running slow then broadly there may be any of the following reason which needs to be check as First Ad.

1. Please check if there are any slow responding threads or threads that are consuming more CPU cycles?

In such scenarios it is better to collect Thread dumps and the CPU metrics to understand the thread level issues/slowness/hung/CPU utilization etc.

https://community.hortonworks.com/articles/72319/how-to-collect-threaddump-using-jcmd-and-analyse-i....

.

2. Please check the memory utilization by your Job and to find out if the memory that is allocated for your job execution is sufficient or needs some tuning. In that case based on the nature of your Job please check the following links:

https://community.hortonworks.com/articles/22419/hive-on-tez-performance-tuning-determining-reducer....
https://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_spark-component-guide/content/ch_tuning-...

.

; ;