Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to specify yarn queue while running "sqoop job --exec jobname"

avatar
 
4 REPLIES 4

avatar

Hi @harsha vardhan

sqoop job -Dmapred.job.queuename=yourqueuename \
--create yourjob \
--etc 

make sure you specify Dmapred.job.queuename directly after 'sqoop job', as this parameter must precede all other arguments.

avatar

Hi @Jonathan Sneep

I already have a sqoop job created with mapred.job.queuename set to default but i'd like to know if it is possible to override that parameter while running "sqoop job --exec jobname" because i tried "sqoop job -Dmapred.job.queuename=queuename --exec jobname" and it didn't work.

avatar

Hi @harsha vardhan Could you explain a bit more on that?

Yes you can override the queue whenever you want. But it also depends on the user/groups access as well. If the user is assigned to specific groups and if the groups are not assigned/given privileges to access any other queue then it will not be possible unless proper access are given to user groups.

But if you have access to multiple queues then , you can have a parameter passed as a queue name to the sqoop job and if the queue name has to be changed, then you can do that with the combination of shell+sqoop.

avatar

Hi @Bala Vignesh N V,

I am sure that its not the issue with permissions because I am able to run other jobs by specifying queues. Is there a problem with the command i use to run ?

I use

sqoop job -Dmapred.job.queuename=queuename --exec jobname

but the job still runs in "default" queue