Created on 09-25-2014 08:15 AM - edited 09-16-2022 02:08 AM
I'd like to be able to have users submit some queries to specific resource pools that will give assurance of dedicated resources for those specific queries.
By default users queries are going into their root.username pool - and that behavior is fine. But once a user has refined their query and we'd like to build an excepted SLA for that query I'd like it to run in a restricted, resource defined pool that I can use to help ensure the resouces that will be dedicated to it.
I have found that mapred.job.queue.name can be used when submitting jobs from command line - but I'd like to be able to do this from Hive/Pig because I mostly have Hive/Pig users.
Thank you.
Created 09-25-2014 09:52 AM
So with some digging I found out how to do this in Hue, it is very easy under the Settings tab when executing a query, just enter the key=mapred.job.queue.name & value = poolname.
It looks like Hue has an interface for this in the Pig editor as well under Hadoop Properties.
From Beeline you can do this like this:
beeline -u jdbc:hive2://[HIVESERVER2HOST]:[PORT]/[DBNAME] -e "[SELECT QUERY];" --hiveconf mapred.job.queue.name=[POOLNAME]
And in Pig the set mapred.job.queue.name [poolname] command can be used.
Created 07-23-2015 06:04 AM
This works for me:
beeline
!connect jdbc:hive2://your.host:your.port/data_base?mapred.job.queue.name=your_queue_name
After that you can execute your queries in beeline