Support Questions

Find answers, ask questions, and share your expertise

how to find long running hadoop/yarn jobs?

how to find long running hadoop/yarn jobs by using command line.

1 ACCEPTED SOLUTION

Hi @kishore sanchina

There is not a straight command to get long running jobs. In Ambari 2.4, they have provided the Zeppelin Dashboard in SmartSense Service(1.3) where we can see all long running jobs and job which has used maximum memory etc.,

Link: https://docs.hortonworks.com/HDPDocuments/SS1/SmartSense-1.3.0/bk_user-guide/content/activity_explor...

Example Screenshot:

12305-longrunningjobslist.png

Option 2:

Prior to that, I have written a bash script which is a length process. Where I will gather all the information from Resource Manager URL using Resource Manager REST API calls and store that information in a CSV file. Then load that data in a CSV file to HDFS and then create a Hive external table on top of it. Then I use to run insert command and move the required columns to the final table and started running simple hive queries to get the list of all long running jobs.

Hope this helps you.

View solution in original post

2 REPLIES 2

Hi @kishore sanchina

There is not a straight command to get long running jobs. In Ambari 2.4, they have provided the Zeppelin Dashboard in SmartSense Service(1.3) where we can see all long running jobs and job which has used maximum memory etc.,

Link: https://docs.hortonworks.com/HDPDocuments/SS1/SmartSense-1.3.0/bk_user-guide/content/activity_explor...

Example Screenshot:

12305-longrunningjobslist.png

Option 2:

Prior to that, I have written a bash script which is a length process. Where I will gather all the information from Resource Manager URL using Resource Manager REST API calls and store that information in a CSV file. Then load that data in a CSV file to HDFS and then create a Hive external table on top of it. Then I use to run insert command and move the required columns to the final table and started running simple hive queries to get the list of all long running jobs.

Hope this helps you.

New Contributor

@Sridhar Reddy,

Could please share the script if possible??