Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Running impala query with specified resosurce pool

Running impala query with specified resosurce pool

Super Collaborator

Hi,

 

I'm using set request_pool= to run an impala query with specific resource pool, but with no success.

 

When i connect to impala-shell and run set request_pool=xxx, I get REQUEST_POOL set to xxx, after running my query i see it still running under the default pool.

 

How i can also pass this if i want to run the impala-shell with variables like impala-shell -i xxxx -q yyyy., Can i add to this commnad the set request_pool.

 

Thanks in advance

9 REPLIES 9

Re: Running impala query with specified resosurce pool

Champion
Make sure that you have the placement rule to allow pools to be specified at runtime. This got me one time and I kept getting the messages that it was set but it would continue to run in the default queue.

As for running it with -q. I haven't tried it but I imaged it would be similar to hive.

impala-shell -i xxxx -q request_pool=new_pool; select...
Highlighted

Re: Running impala query with specified resosurce pool

Super Collaborator
Sure i have a placement rule:

Use the pool specified at run time and create the pool if it does not exist.

Also me getting that the pool set but the queries still running at the
default pool.

Did you get it resolved?

Re: Running impala query with specified resosurce pool

Champion
My issue was that I didn't have that placement rule in place. I created the placement rule to allow it to be specified at runtime and then my queries were assigned to the correct queue.

Is that rule above (have a lower number) than the default pool rule?

The other item to check is that the user you are running the queries from has access to submit to the pool.

Re: Running impala query with specified resosurce pool

Super Collaborator
No.

This is the higher placement rule:

1

Use the pool specified at runtime and create the pool if it does not exist.
Edit
2

Use the pool root.[username], only if the pool exists.
Edit
3

Use the pool root.default.

Re: Running impala query with specified resosurce pool

Super Collaborator

Seems also this option is not working at all:

 

impala-shell -i xxxx -q request_pool=new_pool; select...

 

tried to run 

 

impala-shell -i xxxx -q request_pool=new_pool;mpala-shell -i xxxx -q select ...

 

impala-shell -i xxxx -q "request_pool=new_pool" " select ..."

 

impala-shell -i xxxx -q "request_pool=new_pool"; -q " select ..."

 

 

 

 

Re: Running impala query with specified resosurce pool

Super Collaborator

My bad: forgot that -q is used for a single query use.

 

I solved the issue by putting the 2 select statement at file and used the -f option.

 

Still investigate the issue of the pool.

Re: Running impala query with specified resosurce pool

New Contributor

my solution is creating a new resource pool with its name same as its user:group.

then set the placement rule: "Use the pool root.[primary group], only if the pool exists."

 

Now looks like impala will choose right resource pool after a user login into impala-shell. it also works at HUE.

 

however, the further test shows the user has same name with pool becomes able to use every pool!!

why?? 

Re: Running impala query with specified resosurce pool

Super Collaborator

Hi @AAS Caan you please share the placement Rules?

 

Seems that the rule of Use the pool root.[username], only if the pool exists has higher priority

Re: Running impala query with specified resosurce pool

New Contributor

The only issue I can see is the way you are writing request_pool, it should be all Caps else it will not set to the specific pool.

 

Kinldy use impala-shell -k -i hostname:portnum -B -q 'set REQUEST_POOL=pool_name;'

 

Hope this will resolve the issue.

 

NOTE: I am considering that the Impala Admission Control is applied and the poolname is the one that is created using Cloudera Manager.