Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

--mapreduce-job-name not working

Solved Go to solution
Highlighted

--mapreduce-job-name not working

Hi All,

 

As per the free-form query concept of sqoop, i am trying to use --mapreduce-job-name option to change the default name of job from queryResult.jar to a name that help me to identify it , but it's not working.

Anyone has suggestion on this?

 

Thanks,

Rupesh

3 ACCEPTED SOLUTIONS

Accepted Solutions

Re: --mapreduce-job-name not working

Champion

Could you let us know what name is it using instead of your --mapreduce-job-name ? 

 

 

1 . Could you enable --verbose  and trying firing a   --mapreduce-job-name in  test sqoop import just to see if its the same across all the sqoop commands. 

 

2. whats your mapreduce-framework.name in mapred-site.xml

Re: --mapreduce-job-name not working

Champion

Can you perform a simple import statement with --mapreduce-name to see if you are able to override the default jar name. 

sqoop import --connect jdbc:mysql://localhost/Logo  --username tuy --password **** --table Golo  --mapreduce-job-name myJOb

 

Re: --mapreduce-job-name not working

Master Guru
Passing --mapreduce-job-name just causes the jar name not to be used as the MR job name, which is Sqoop's default behaviour. The inverse doesn't hold true (i.e. the generated jar name does not change).
9 REPLIES 9

Re: --mapreduce-job-name not working

Champion

Could you let us know what name is it using instead of your --mapreduce-job-name ? 

 

 

1 . Could you enable --verbose  and trying firing a   --mapreduce-job-name in  test sqoop import just to see if its the same across all the sqoop commands. 

 

2. whats your mapreduce-framework.name in mapred-site.xml

Re: --mapreduce-job-name not working

a. I am using the below command. Where i want to change the name of the job from QueryResult.jar to rupesh.jar . but its not happening. b. framework is yarn in the mapred-site.xml sqoop import --connect jdbc:mysql://localhost/retail_db --username **** --password ******* --query "SELECT O1.order_id,\ O1.order_date, \ O1.order_customer_id, \ O2.order_item_order_id, \ O2.order_item_product_id, \ O2.order_item_quantity \ FROM orders as O1 \ JOIN order_items as O2 ON(O1.order_id=O2.order_item_id) \ WHERE \$CONDITIONS AND O1.order_id <=3000" --split-by O1.order_id --target-dir /user/cloudera/rupesh7 --mapreduce-job-name rupesh upon executing the above query getting below message. 17/03/20 21:26:52 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/65c368533cccb526215d289ef0bb62c7/QueryResult.jar 17/03/20 21:26:52 INFO mapreduce.ImportJobBase: Beginning query import.

Re: --mapreduce-job-name not working

Champion

Can you perform a simple import statement with --mapreduce-name to see if you are able to override the default jar name. 

sqoop import --connect jdbc:mysql://localhost/Logo  --username tuy --password **** --table Golo  --mapreduce-job-name myJOb

 

Re: --mapreduce-job-name not working

Still it's not working. if i am importing a single table, the jar is created with tablename.jar and in case of free-form-query QueryResult.jar is coming.
But i am not able to get the name that i am providing using --mapreduce-job-name

Re: --mapreduce-job-name not working

Champion

could you use --verbose in your import and paste the logs. 

 

Re: --mapreduce-job-name not working

Thanks... i am able to see the name now. Actually i was checking the logs of the CLI rather than the log at HUE.

Re: --mapreduce-job-name not working

Champion

great. Just curious to know if it reflecting the same in Resource manager Web Ui 

Re: --mapreduce-job-name not working

Master Guru
Passing --mapreduce-job-name just causes the jar name not to be used as the MR job name, which is Sqoop's default behaviour. The inverse doesn't hold true (i.e. the generated jar name does not change).

Re: --mapreduce-job-name not working

This is the perfect answer i was looking for. Thanks Harsh.