Sqoop job asking password for every second time running which is scheduled via Oozie

[#1]

Hi,

I'm Trying to create a Oozie workflow for a sqoop incremental job loading data from Mysql to Hive External table.

when im trying to run the same job via command line even for n times it's not asking for password, but when i'm configure and execute the same job via Oozie workflow(via hue interface) first time its running successfully but from second time onwords it is asking for the Mysql Password which i passed already when creating the Job with the parameter --password 'MYPassword'

is there any workaround for this issue ?

i have altered the sqoop-site.xml as follows

<property>

<name>sqoop.metastore.client.autoconnect.url</name>

<value>--meta-connect jdbc:hsqldb:hsql://localhost:16000/sqoop</value>

<description>The connect string to use when connecting to a job-management metastore. If unspecified, uses ~/.sqoop/. You can specify a different path here. </description>

</property>

<property>

<name>sqoop.metastore.client.autoconnect.username</name>

<value>SA</value>

<description>The username to bind to the metastore. </description>

</property>

<property>

<name>sqoop.metastore.client.autoconnect.password</name>

<value></value>

<description>The password to bind to the metastore. </description>

</property>

<property>

<name>sqoop.metastore.client.record.password</name>

<value>true</value>

<description>If true, allow saved passwords in the metastore. </description>

</property>

<property>

<name>sqoop.metastore.server.location</name>

<value>/tmp/sqoop-metastore/shared.db</value>

<description>Path to the shared metastore database files. If this is not set, it will be placed in ~/.sqoop/. </description> </property>

<property>

<name>sqoop.metastore.server.port</name>

<value>16000</value>

<description>Port that this metastore should listen on. </description>

</property>

when i'm trying to run second time it was throwing the following Error:

 14614 [uber-SubtaskRunner] WARN  org.apache.sqoop.tool.SqoopTool  - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
  14789 [uber-SubtaskRunner] INFO  org.apache.sqoop.Sqoop  - Running Sqoop version: 1.4.5-cdh5.4.2
  15587 [uber-SubtaskRunner] ERROR org.apache.sqoop.SqoopOptions  - It seems that you have launched a Sqoop metastore job via
  15588 [uber-SubtaskRunner] ERROR org.apache.sqoop.SqoopOptions  - Oozie with sqoop.metastore.client.record.password disabled.
  15588 [uber-SubtaskRunner] ERROR org.apache.sqoop.SqoopOptions  - But this configuration is not supported because Sqoop can't
  15588 [uber-SubtaskRunner] ERROR org.apache.sqoop.SqoopOptions  - prompt the user to enter the password while being executed
  15588 [uber-SubtaskRunner] ERROR org.apache.sqoop.SqoopOptions  - as Oozie tasks. Please enable sqoop.metastore.client.record
  15588 [uber-SubtaskRunner] ERROR org.apache.sqoop.SqoopOptions  - .password in sqoop-site.xml, or provide the password
  15588 [uber-SubtaskRunner] ERROR org.apache.sqoop.SqoopOptions  - explicitly using --password in the command tag of the Oozie
  15588 [uber-SubtaskRunner] ERROR org.apache.sqoop.SqoopOptions  - workflow file.
  15650 [uber-SubtaskRunner] WARN  org.apache.sqoop.ConnFactory  - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
  15831 [uber-SubtaskRunner] INFO  org.apache.sqoop.manager.MySQLManager  - Preparing to use a MySQL streaming resultset.
  15831 [uber-SubtaskRunner] INFO  org.apache.sqoop.tool.CodeGenTool  - Beginning code generation
  16113 [uber-SubtaskRunner] ERROR org.apache.sqoop.manager.SqlManager  - Error executing statement: java.sql.SQLException: Access denied for user 'root'@'localhost' (using password: NO)
  java.sql.SQLException: Access denied for user 'root'@'localhost' (using password: NO)

[#2]

i have found that db.password property for the job was not working fine when the job was created with with password as follows

bin/sqoop job --meta-connect jdbc:hsqldb:hsql://localhost:16000/sqoop --create inc_imp_patient_hive -- import --connect jdbc:mysql://localhost/test --username root --password cloudera --table patient --check-column pid --incremental append --last-value 0 --target-dir /Sqoop/Output/hdfs/patient_Hive -m 1

it shows the db.password=cloudera in job --show job_name

after executing it through Oozie workflow it was showing the db.password =

nothing is the in the db.password place. how to resolve this issue ?

[#3]

worked fine after alter the create job statement with --password-file as follows:

bin/sqoop job --meta-connect jdbc:hsqldb:hsql://localhost:16000/sqoop --create inc_imp_patient_hive -- import --connect jdbc:mysql://localhost/test --username root --password-file /mysql_pwd.pwd --table patient --check-column pid --incremental append --last-value 0 --target-dir /Sqoop/Output/hdfs/patient_Hive -m 1

[#4]

@Manikandan Jeyabal

Glad to hear it worked with --password-file, you could also try updating the password in sqoop-site.xml under 'sqoop.metastore.client.autoconnect.password' and try.