Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

--mapreduce-job-name not working

avatar

Hi All,

 

As per the free-form query concept of sqoop, i am trying to use --mapreduce-job-name option to change the default name of job from queryResult.jar to a name that help me to identify it , but it's not working.

Anyone has suggestion on this?

 

Thanks,

Rupesh

3 ACCEPTED SOLUTIONS

avatar
Champion

Could you let us know what name is it using instead of your --mapreduce-job-name ? 

 

 

1 . Could you enable --verbose  and trying firing a   --mapreduce-job-name in  test sqoop import just to see if its the same across all the sqoop commands. 

 

2. whats your mapreduce-framework.name in mapred-site.xml

View solution in original post

avatar
Champion

Can you perform a simple import statement with --mapreduce-name to see if you are able to override the default jar name. 

sqoop import --connect jdbc:mysql://localhost/Logo  --username tuy --password **** --table Golo  --mapreduce-job-name myJOb

 

View solution in original post

avatar
Mentor
Passing --mapreduce-job-name just causes the jar name not to be used as the MR job name, which is Sqoop's default behaviour. The inverse doesn't hold true (i.e. the generated jar name does not change).

View solution in original post

9 REPLIES 9

avatar
Champion

Could you let us know what name is it using instead of your --mapreduce-job-name ? 

 

 

1 . Could you enable --verbose  and trying firing a   --mapreduce-job-name in  test sqoop import just to see if its the same across all the sqoop commands. 

 

2. whats your mapreduce-framework.name in mapred-site.xml

avatar
a. I am using the below command. Where i want to change the name of the job from QueryResult.jar to rupesh.jar . but its not happening. b. framework is yarn in the mapred-site.xml sqoop import --connect jdbc:mysql://localhost/retail_db --username **** --password ******* --query "SELECT O1.order_id,\ O1.order_date, \ O1.order_customer_id, \ O2.order_item_order_id, \ O2.order_item_product_id, \ O2.order_item_quantity \ FROM orders as O1 \ JOIN order_items as O2 ON(O1.order_id=O2.order_item_id) \ WHERE \$CONDITIONS AND O1.order_id <=3000" --split-by O1.order_id --target-dir /user/cloudera/rupesh7 --mapreduce-job-name rupesh upon executing the above query getting below message. 17/03/20 21:26:52 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/65c368533cccb526215d289ef0bb62c7/QueryResult.jar 17/03/20 21:26:52 INFO mapreduce.ImportJobBase: Beginning query import.

avatar
Champion

Can you perform a simple import statement with --mapreduce-name to see if you are able to override the default jar name. 

sqoop import --connect jdbc:mysql://localhost/Logo  --username tuy --password **** --table Golo  --mapreduce-job-name myJOb

 

avatar
Still it's not working. if i am importing a single table, the jar is created with tablename.jar and in case of free-form-query QueryResult.jar is coming.
But i am not able to get the name that i am providing using --mapreduce-job-name

avatar
Champion

could you use --verbose in your import and paste the logs. 

 

avatar

Thanks... i am able to see the name now. Actually i was checking the logs of the CLI rather than the log at HUE.

avatar
Champion

great. Just curious to know if it reflecting the same in Resource manager Web Ui 

avatar
Mentor
Passing --mapreduce-job-name just causes the jar name not to be used as the MR job name, which is Sqoop's default behaviour. The inverse doesn't hold true (i.e. the generated jar name does not change).

avatar

This is the perfect answer i was looking for. Thanks Harsh.