Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to find long running hadoop/yarn jobs?

Solved Go to solution

how to find long running hadoop/yarn jobs?

how to find long running hadoop/yarn jobs by using command line.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: how to find long running hadoop/yarn jobs?

Hi @kishore sanchina

There is not a straight command to get long running jobs. In Ambari 2.4, they have provided the Zeppelin Dashboard in SmartSense Service(1.3) where we can see all long running jobs and job which has used maximum memory etc.,

Link: https://docs.hortonworks.com/HDPDocuments/SS1/SmartSense-1.3.0/bk_user-guide/content/activity_explor...

Example Screenshot:

12305-longrunningjobslist.png

Option 2:

Prior to that, I have written a bash script which is a length process. Where I will gather all the information from Resource Manager URL using Resource Manager REST API calls and store that information in a CSV file. Then load that data in a CSV file to HDFS and then create a Hive external table on top of it. Then I use to run insert command and move the required columns to the final table and started running simple hive queries to get the list of all long running jobs.

Hope this helps you.

View solution in original post

2 REPLIES 2
Highlighted

Re: how to find long running hadoop/yarn jobs?

Hi @kishore sanchina

There is not a straight command to get long running jobs. In Ambari 2.4, they have provided the Zeppelin Dashboard in SmartSense Service(1.3) where we can see all long running jobs and job which has used maximum memory etc.,

Link: https://docs.hortonworks.com/HDPDocuments/SS1/SmartSense-1.3.0/bk_user-guide/content/activity_explor...

Example Screenshot:

12305-longrunningjobslist.png

Option 2:

Prior to that, I have written a bash script which is a length process. Where I will gather all the information from Resource Manager URL using Resource Manager REST API calls and store that information in a CSV file. Then load that data in a CSV file to HDFS and then create a Hive external table on top of it. Then I use to run insert command and move the required columns to the final table and started running simple hive queries to get the list of all long running jobs.

Hope this helps you.

View solution in original post

Highlighted

Re: how to find long running hadoop/yarn jobs?

New Contributor

@Sridhar Reddy,

Could please share the script if possible??

Don't have an account?
Coming from Hortonworks? Activate your account here