Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Running impala query with specified resosurce pool

avatar
Master Collaborator

Hi,

 

I'm using set request_pool= to run an impala query with specific resource pool, but with no success.

 

When i connect to impala-shell and run set request_pool=xxx, I get REQUEST_POOL set to xxx, after running my query i see it still running under the default pool.

 

How i can also pass this if i want to run the impala-shell with variables like impala-shell -i xxxx -q yyyy., Can i add to this commnad the set request_pool.

 

Thanks in advance

9 REPLIES 9

avatar
Champion
Make sure that you have the placement rule to allow pools to be specified at runtime. This got me one time and I kept getting the messages that it was set but it would continue to run in the default queue.

As for running it with -q. I haven't tried it but I imaged it would be similar to hive.

impala-shell -i xxxx -q request_pool=new_pool; select...

avatar
Master Collaborator
Sure i have a placement rule:

Use the pool specified at run time and create the pool if it does not exist.

Also me getting that the pool set but the queries still running at the
default pool.

Did you get it resolved?

avatar
Champion
My issue was that I didn't have that placement rule in place. I created the placement rule to allow it to be specified at runtime and then my queries were assigned to the correct queue.

Is that rule above (have a lower number) than the default pool rule?

The other item to check is that the user you are running the queries from has access to submit to the pool.

avatar
Master Collaborator
No.

This is the higher placement rule:

1

Use the pool specified at runtime and create the pool if it does not exist.
Edit
2

Use the pool root.[username], only if the pool exists.
Edit
3

Use the pool root.default.

avatar
Master Collaborator

Seems also this option is not working at all:

 

impala-shell -i xxxx -q request_pool=new_pool; select...

 

tried to run 

 

impala-shell -i xxxx -q request_pool=new_pool;mpala-shell -i xxxx -q select ...

 

impala-shell -i xxxx -q "request_pool=new_pool" " select ..."

 

impala-shell -i xxxx -q "request_pool=new_pool"; -q " select ..."

 

 

 

 

avatar
Master Collaborator

My bad: forgot that -q is used for a single query use.

 

I solved the issue by putting the 2 select statement at file and used the -f option.

 

Still investigate the issue of the pool.

avatar
New Contributor

my solution is creating a new resource pool with its name same as its user:group.

then set the placement rule: "Use the pool root.[primary group], only if the pool exists."

 

Now looks like impala will choose right resource pool after a user login into impala-shell. it also works at HUE.

 

however, the further test shows the user has same name with pool becomes able to use every pool!!

why?? 

avatar
Master Collaborator

Hi @AAS Caan you please share the placement Rules?

 

Seems that the rule of Use the pool root.[username], only if the pool exists has higher priority

avatar
New Contributor

The only issue I can see is the way you are writing request_pool, it should be all Caps else it will not set to the specific pool.

 

Kinldy use impala-shell -k -i hostname:portnum -B -q 'set REQUEST_POOL=pool_name;'

 

Hope this will resolve the issue.

 

NOTE: I am considering that the Impala Admission Control is applied and the poolname is the one that is created using Cloudera Manager.