Get actual query details of all HIVE jobs in last 30 days

[#1]

Hi All

I am trying to get all hive queries for last 30 days ran on my cluster.

I selected all the jobs on cloudera YARN application UI and filtered for last 30 days and also selected attribute of hive_query_string, which allows me to see the actual query.

The only issue is that cloudera restricts UI to show only last 100 jobs at a time.

Because of this i can't get details on all the other queries.

I tried to hit this api http://cluster:8088/ws/v1/cluster/apps to get all the details, two issue: there is no filter and other is there is no hive_query_string variable to filter. It just shows me all the job details.

Is there an API exposed by cloudera where i can filter and get all the hive_query_string or if i can configure the cloudera UI to show me more than 100 jobs(or just export it).

Let me know.

Thanks

AB

[#2 · Accepted solution]

The below link connects and filter results from API to get last 30 days of jobs and limit is increased to 10000 jobs at a time.

Note: All times "endTime=1503438687495" are EPOC time, so filter your times based on requirement.

Also set limit to what ever number of jobs you want to be displayed.

http://cloudera-manager-host-ip/cmf/yarn/completedApplications?startTime=1500758462000&endTime=1503438687495&offset=0&limit=10000&serviceName=yarn&histogramAttributes=allocated_memory_seconds%2Callocated_vcore_seconds%2Ccpu_milliseconds%2Capplication_duration%2Chdfs_bytes_read%2Chdfs_bytes_written