question Re: Password Encryption in Sqoop Job not working in Archives of Support Questions (Read Only)

Password Encryption in Sqoop Job not working

reddy_gayathri — Sun, 18 Aug 2019 11:53:37 GMT

I am using password encryption method in Sqoop job for data ingestion into Hadoop. Used Dhadoop.security.credential.provider.path to encrypt the password.

But when I try to create the Sqoop job in CLI, it is unable to parse the arguments. Below is the code I used and error I got also mentioned below.

CODE

sqoop job --create password-test --meta-connect jdbc:hsqldb:hsql://<hostname>:<port>/sqoop -- import -Dhadoop.security.credential.provider.path=jceks://hdfs/user/<username>/<username>.password.jceks --connect "jdbc:oracle:thin:<hostname>:<Port>:<sid>" --username <username> --table <tablename> --password-alias <password-alias-name> --fields-terminated-by '\001' --null-string '\N' --null-non-string '\N' --lines-terminated-by '\n' --target-dir '/user/<username>/<staging loc>' --incremental append --check-column <colname> --last-value <value> --num-mappers 8

ERROR

Re: Password Encryption in Sqoop Job not working

njayakumar — Tue, 20 Sep 2016 17:54:31 GMT

@Gayathri Reddy G - pass generic arguments like -D after SQOOP JOB -Dhadoop.security.credential.provider.path=jceks ....

General syntax is

sqoop-job (generic-args) (job-args) [-- [subtool-name] (subtool-args)]

Re: Password Encryption in Sqoop Job not working

njayakumar — Tue, 20 Sep 2016 17:58:29 GMT

You must supply the generic arguments -conf, -D, and so on after the tool name but before any tool-specific arguments (such as --connect). Note that generic Hadoop arguments are preceeded by a single dash character (-), whereas tool-specific arguments start with two dashes (--), unless they are single character arguments such as -P.

https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_using_generic_and_specific_arguments

Re: Password Encryption in Sqoop Job not working

reddy_gayathri — Tue, 20 Sep 2016 18:40:44 GMT

@njayakumar passed the Generic arguments first to the sqoop job, now its working fine. Thanks