Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to specify yarn queue while running "sqoop job --exec jobname"

How to specify yarn queue while running "sqoop job --exec jobname"

New Contributor
 
4 REPLIES 4

Re: How to specify yarn queue while running "sqoop job --exec jobname"

Hi @harsha vardhan

sqoop job -Dmapred.job.queuename=yourqueuename \
--create yourjob \
--etc 

make sure you specify Dmapred.job.queuename directly after 'sqoop job', as this parameter must precede all other arguments.

Re: How to specify yarn queue while running "sqoop job --exec jobname"

New Contributor

Hi @Jonathan Sneep

I already have a sqoop job created with mapred.job.queuename set to default but i'd like to know if it is possible to override that parameter while running "sqoop job --exec jobname" because i tried "sqoop job -Dmapred.job.queuename=queuename --exec jobname" and it didn't work.

Re: How to specify yarn queue while running "sqoop job --exec jobname"

Hi @harsha vardhan Could you explain a bit more on that?

Yes you can override the queue whenever you want. But it also depends on the user/groups access as well. If the user is assigned to specific groups and if the groups are not assigned/given privileges to access any other queue then it will not be possible unless proper access are given to user groups.

But if you have access to multiple queues then , you can have a parameter passed as a queue name to the sqoop job and if the queue name has to be changed, then you can do that with the combination of shell+sqoop.

Re: How to specify yarn queue while running "sqoop job --exec jobname"

New Contributor

Hi @Bala Vignesh N V,

I am sure that its not the issue with permissions because I am able to run other jobs by specifying queues. Is there a problem with the command i use to run ?

I use

sqoop job -Dmapred.job.queuename=queuename --exec jobname

but the job still runs in "default" queue