Reply
Highlighted
Explorer
Posts: 25
Registered: ‎04-26-2016

How to kill Long running queries on hive or impala via hue editor , is there a way to set a policy

How to kill or cancel a query if it is running longer than normal automatically, i killed queries that are running longer from cloudera manager. but want something automated way.

 

Please provide details if there is a possibility.

 

Thank you very much for the helpful info.

 

Posts: 642
Topics: 3
Kudos: 121
Solutions: 67
Registered: ‎08-16-2016

Re: How to kill Long running queries on hive or impala via hue editor , is there a way to set a poli

I don't know of any way. Hadoop in general doesn't care how long it takes; it is more concerned an auto-recover of the platform so that jobs can finish no matter what. You can limit the number of queries or jobs by user or group, you can limit the resources to users or groups. I just don't think there is a way to automatically kill jobs or queries running longer than X. I know other products, like Pepperdata, can track and alert you. It still require manually intervention.

Can we step back and you explain what your issue is with long running jobs? As maybe the root cause can be addressed there so job do not run for so long or hold back others.
Champion
Posts: 776
Registered: ‎05-16-2016

Re: How to kill Long running queries on hive or impala via hue editor , is there a way to set a poli

As far I am concerned I dont think there is no way to dectect them kill automatically . 

One way to slove the long runining query is to inspect your query flow , streaming also enable 

strict mode - In this mode some risky query will not be performed like cartesian product , ordering without limit clause. 

Explorer
Posts: 25
Registered: ‎04-26-2016

Re: How to kill Long running queries on hive or impala via hue editor , is there a way to set a poli

Hi csguna,

Thanks for the helpful tip, can you pls tell me how to enable strict mode on impala queries.

Thank you.
Champion
Posts: 776
Registered: ‎05-16-2016

Re: How to kill Long running queries on hive or impala via hue editor , is there a way to set a poli

There is no direct way of enforcing a strict mode to Impala 

However you can cotrol the run time behaviour of the queries by issuing SET statement 

for example controling the memory limit , max_scan_range_lenth

 

Please refer this link 

SET STATEMENT

Explorer
Posts: 7
Registered: ‎12-08-2017

Re: How to kill Long running queries on hive or impala via hue editor , is there a way to set a poli

I use the below trick in shell scripts for known queries that don't have an impact if killed. 

 

I call the impala-shell from bash/cron using the below line

 

timeout 600 impala-shell -i $IMPALA_ADDRESS -q "compute stats hive_db.table_name;" 

 

The all important part is "timeout 600" which will end the session after 10 mins. 

Announcements