Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Changing Impala-shell session timeout

Changing Impala-shell session timeout

Explorer

I'm executing 300+ queries at once via impala-shell in a pool that allows a queue of 500. However, after 10 minutes I start to see some queries flagged as error with a query status of "session closed". The query uses the following option "SET IDLE_SESSION_TIMEOUT=3600;" Any thoughts on what property I need to change so the session stays open longer than 10 minutes?

 

I'm running Impala Shell v2.12.0-cdh5.15.2

5 REPLIES 5

Re: Changing Impala-shell session timeout

Master Collaborator

Sessions with queries that are queued in admission control generally shouldn't be considered be considered idle. What Impala server version are you connected to?


What does your script look like? Are you running 300+ instances of impala-shell at the same time?

Re: Changing Impala-shell session timeout

Explorer

I'm connected to... 

impalad version 2.12.0-cdh5.15.2

 

And yes, I'm running 300+ instances of impala-shell at the same time. I have a python script that loops through a set of queries and executes one impala-shell command per query:

impala-shell -k -i edpprd --ssl --quiet -B -c -q "SET REQUEST_POOL=root.pool1; SET IDLE_SESSION_TIMEOUT=3600; select 'table1' as table_name, count(1) as cloud_count from default.table"

 

Some of the queries take 30+ minutes to complete when running by themselves. I also tried this without defining a pool and just submitting all the queries on their own and it overwhelms the impala server memory and results in 100+ failed queries. 

Re: Changing Impala-shell session timeout

Guru
Hi @EricC

Are you able to share some of the failed impala query PROFILE for review? It might tell some story from there.

Cheers
EricL

Re: Changing Impala-shell session timeout

Explorer

I ended up submitting this as a Cloudera Support request. I will post the solution once we figure it out.

Highlighted

Re: Changing Impala-shell session timeout

Master Collaborator

So IDLE_SESSION_TIMEOUT doesn't actually do anything if you set it in impala-shell. There's a technical distinction here - "set" in impala-shell is implemented in the shell itself, then send along with the queries. Whereas if you run "set" as a statement through, say, the JDBC driver, then "set" is run as a server-side command that modifies the session state.

 

I tried to reproduce the issue on my local Impala environment, but couldn't. I was watching the /sessions page in the debug page and I could see the "last accessed" time get continually incremented while the query was in the queue. I think any timeout issue probably got indirectly fixed by https://issues.apache.org/jira/browse/IMPALA-5216 in CDH6.1/Impala 3.1, since the query state would be polled directly by impala-shell while the query is in the queue, rather than having impala-shell blocked waiting for a response.

 

Thanks for submitting the support request, that should help get to the bottom of this.

Don't have an account?
Coming from Hortonworks? Activate your account here