Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Get actual query details of all HIVE jobs in last 30 days

avatar
Contributor

Hi All

I am trying to get all hive queries for last 30 days ran on my cluster.

I selected all the jobs on cloudera YARN application UI and filtered for last 30 days and also selected attribute of hive_query_string, which allows me to see the actual query.

The only issue is that cloudera restricts UI to show only last 100 jobs at a time.

Because of this i can't get details on all the other queries.

I tried to hit this api http://cluster:8088/ws/v1/cluster/apps to get all the details, two issue: there is no filter and other is there is no hive_query_string variable to filter. It just shows me all the job details.

Is there an API exposed by cloudera where i can filter and get all the hive_query_string or if i can configure the cloudera UI to show me more than 100 jobs(or just export it).

 

Let me know.

 

Thanks

AB

Abhishek
1 ACCEPTED SOLUTION

avatar
Contributor

The below link connects and filter results from API to get last 30 days of jobs and limit is increased to 10000 jobs at a time.

Note: All times "endTime=1503438687495" are EPOC time, so filter your times based on requirement.

Also set limit to what ever number of jobs you want to be displayed.

 

http://cloudera-manager-host-ip/cmf/yarn/completedApplications?startTime=1500758462000&endTime=1503438687495&offset=0&limit=10000&serviceName=yarn&histogramAttributes=allocated_memory_seconds%2Callocated_vcore_seconds%2Ccpu_milliseconds%2Capplication_duration%2Chdfs_bytes_read%2Chdfs_bytes_written

Abhishek

View solution in original post

1 REPLY 1

avatar
Contributor

The below link connects and filter results from API to get last 30 days of jobs and limit is increased to 10000 jobs at a time.

Note: All times "endTime=1503438687495" are EPOC time, so filter your times based on requirement.

Also set limit to what ever number of jobs you want to be displayed.

 

http://cloudera-manager-host-ip/cmf/yarn/completedApplications?startTime=1500758462000&endTime=1503438687495&offset=0&limit=10000&serviceName=yarn&histogramAttributes=allocated_memory_seconds%2Callocated_vcore_seconds%2Ccpu_milliseconds%2Capplication_duration%2Chdfs_bytes_read%2Chdfs_bytes_written

Abhishek