question Re: --mapreduce-job-name not working in Archives of Support Questions (Read Only)

--mapreduce-job-name not working

rupeshverma88 — Fri, 16 Sep 2022 11:14:09 GMT

Hi All,

As per the free-form query concept of sqoop, i am trying to use --mapreduce-job-name option to change the default name of job from queryResult.jar to a name that help me to identify it , but it's not working.

Anyone has suggestion on this?

Thanks,

Rupesh

Re: --mapreduce-job-name not working

csguna — Fri, 17 Mar 2017 06:05:17 GMT

Could you let us know what name is it using instead of your --mapreduce-job-name ?

1 . Could you enable --verbose and trying firing a --mapreduce-job-name in test sqoop import just to see if its the same across all the sqoop commands.

2. whats your mapreduce-framework.name in mapred-site.xml

Re: --mapreduce-job-name not working

rupeshverma88 — Tue, 21 Mar 2017 06:03:46 GMT

a. I am using the below command. Where i want to change the name of the job from QueryResult.jar to rupesh.jar . but its not happening. b. framework is yarn in the mapred-site.xml sqoop import --connect jdbc:mysql://localhost/retail_db --username **** --password ******* --query "SELECT O1.order_id,\ O1.order_date, \ O1.order_customer_id, \ O2.order_item_order_id, \ O2.order_item_product_id, \ O2.order_item_quantity \ FROM orders as O1 \ JOIN order_items as O2 ON(O1.order_id=O2.order_item_id) \ WHERE \$CONDITIONS AND O1.order_id <=3000" --split-by O1.order_id --target-dir /user/cloudera/rupesh7 --mapreduce-job-name rupesh upon executing the above query getting below message. 17/03/20 21:26:52 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/65c368533cccb526215d289ef0bb62c7/QueryResult.jar 17/03/20 21:26:52 INFO mapreduce.ImportJobBase: Beginning query import.

Re: --mapreduce-job-name not working

csguna — Tue, 21 Mar 2017 07:08:40 GMT

Can you perform a simple import statement with --mapreduce-name to see if you are able to override the default jar name.

sqoop import --connect jdbc:mysql://localhost/Logo  --username tuy --password **** --table Golo  --mapreduce-job-name myJOb

Re: --mapreduce-job-name not working

rupeshverma88 — Tue, 21 Mar 2017 18:35:27 GMT

Still it's not working. if i am importing a single table, the jar is created with tablename.jar and in case of free-form-query QueryResult.jar is coming.
But i am not able to get the name that i am providing using --mapreduce-job-name

Re: --mapreduce-job-name not working

csguna — Wed, 22 Mar 2017 07:40:47 GMT

could you use --verbose in your import and paste the logs.

Re: --mapreduce-job-name not working

rupeshverma88 — Wed, 22 Mar 2017 18:37:41 GMT

Thanks... i am able to see the name now. Actually i was checking the logs of the CLI rather than the log at HUE.

Re: --mapreduce-job-name not working

csguna — Thu, 23 Mar 2017 02:50:23 GMT

great. Just curious to know if it reflecting the same in Resource manager Web Ui

Re: --mapreduce-job-name not working

Harsh J — Thu, 23 Mar 2017 04:04:04 GMT

Passing --mapreduce-job-name just causes the jar name not to be used as the MR job name, which is Sqoop's default behaviour. The inverse doesn't hold true (i.e. the generated jar name does not change).

Re: --mapreduce-job-name not working

rupeshverma88 — Thu, 23 Mar 2017 04:57:29 GMT

This is the perfect answer i was looking for. Thanks Harsh.