Reply
Highlighted
Explorer
Posts: 8
Registered: ‎01-08-2017
Accepted Solution

--mapreduce-job-name not working

Hi All,

 

As per the free-form query concept of sqoop, i am trying to use --mapreduce-job-name option to change the default name of job from queryResult.jar to a name that help me to identify it , but it's not working.

Anyone has suggestion on this?

 

Thanks,

Rupesh

Champion
Posts: 543
Registered: ‎05-16-2016

Re: --mapreduce-job-name not working

Could you let us know what name is it using instead of your --mapreduce-job-name ? 

 

 

1 . Could you enable --verbose  and trying firing a   --mapreduce-job-name in  test sqoop import just to see if its the same across all the sqoop commands. 

 

2. whats your mapreduce-framework.name in mapred-site.xml

Explorer
Posts: 8
Registered: ‎01-08-2017

Re: --mapreduce-job-name not working

a. I am using the below command. Where i want to change the name of the job from QueryResult.jar to rupesh.jar . but its not happening. b. framework is yarn in the mapred-site.xml sqoop import --connect jdbc:mysql://localhost/retail_db --username **** --password ******* --query "SELECT O1.order_id,\ O1.order_date, \ O1.order_customer_id, \ O2.order_item_order_id, \ O2.order_item_product_id, \ O2.order_item_quantity \ FROM orders as O1 \ JOIN order_items as O2 ON(O1.order_id=O2.order_item_id) \ WHERE \$CONDITIONS AND O1.order_id <=3000" --split-by O1.order_id --target-dir /user/cloudera/rupesh7 --mapreduce-job-name rupesh upon executing the above query getting below message. 17/03/20 21:26:52 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/65c368533cccb526215d289ef0bb62c7/QueryResult.jar 17/03/20 21:26:52 INFO mapreduce.ImportJobBase: Beginning query import.
Champion
Posts: 543
Registered: ‎05-16-2016

Re: --mapreduce-job-name not working

Can you perform a simple import statement with --mapreduce-name to see if you are able to override the default jar name. 

sqoop import --connect jdbc:mysql://localhost/Logo  --username tuy --password **** --table Golo  --mapreduce-job-name myJOb

 

Explorer
Posts: 8
Registered: ‎01-08-2017

Re: --mapreduce-job-name not working

Still it's not working. if i am importing a single table, the jar is created with tablename.jar and in case of free-form-query QueryResult.jar is coming.
But i am not able to get the name that i am providing using --mapreduce-job-name
Champion
Posts: 543
Registered: ‎05-16-2016

Re: --mapreduce-job-name not working

could you use --verbose in your import and paste the logs. 

 

Explorer
Posts: 8
Registered: ‎01-08-2017

Re: --mapreduce-job-name not working

Thanks... i am able to see the name now. Actually i was checking the logs of the CLI rather than the log at HUE.

Champion
Posts: 543
Registered: ‎05-16-2016

Re: --mapreduce-job-name not working

great. Just curious to know if it reflecting the same in Resource manager Web Ui 

Posts: 1,524
Kudos: 265
Solutions: 232
Registered: ‎07-31-2013

Re: --mapreduce-job-name not working

Passing --mapreduce-job-name just causes the jar name not to be used as the MR job name, which is Sqoop's default behaviour. The inverse doesn't hold true (i.e. the generated jar name does not change).
Backline Customer Operations Engineer
Explorer
Posts: 8
Registered: ‎01-08-2017

Re: --mapreduce-job-name not working

This is the perfect answer i was looking for. Thanks Harsh.

Announcements