Get actual query details of all HIVE jobs in last 30 days

ABaaya — Tue, 22 Aug 2017 20:40:14 GMT

Hi All

I am trying to get all hive queries for last 30 days ran on my cluster.

I selected all the jobs on cloudera YARN application UI and filtered for last 30 days and also selected attribute of hive_query_string, which allows me to see the actual query.

The only issue is that cloudera restricts UI to show only last 100 jobs at a time.

Because of this i can't get details on all the other queries.

I tried to hit this api http://cluster:8088/ws/v1/cluster/apps to get all the details, two issue: there is no filter and other is there is no hive_query_string variable to filter. It just shows me all the job details.

Is there an API exposed by cloudera where i can filter and get all the hive_query_string or if i can configure the cloudera UI to show me more than 100 jobs(or just export it).

Let me know.

Thanks

Re: Get actual query details of all HIVE jobs in last 30 days

ABaaya — Thu, 24 Aug 2017 14:32:02 GMT

The below link connects and filter results from API to get last 30 days of jobs and limit is increased to 10000 jobs at a time.

Note: All times "endTime=1503438687495" are EPOC time, so filter your times based on requirement.

Also set limit to what ever number of jobs you want to be displayed.

http://cloudera-manager-host-ip/cmf/yarn/completedApplications?startTime=1500758462000&endTime=1503438687495&offset=0&limit=10000&serviceName=yarn&histogramAttributes=allocated_memory_seconds%2Callocated_vcore_seconds%2Ccpu_milliseconds%2Capplication_duration%2Chdfs_bytes_read%2Chdfs_bytes_written

question Get actual query details of all HIVE jobs in last 30 days in Archives of Support Questions (Read Only)

Get actual query details of all HIVE jobs in last 30 days

Re: Get actual query details of all HIVE jobs in last 30 days