Hi,   You can use couple of parameters for strings columns and non string column.   ## string columns:- --null-string   ## non string columns:- --null-non-string   Link:- https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_null_string_handling   Above is the link will help you to handle these characters.   Regards Nitish ... View more

Hi,   Please note that this error generally happens when you are trying to query the object which is no longer avaialable.   Link:- http://www.dba-oracle.com/t_ora_08103_object_no_longer_exists.htm   I would request you to go above the link which shares more details on this that when this error will happen.   Regards Nitish ... View more

Hi,   Can you please upload the full Sqoop command and the console level logs so we can test it out.   Regards Nitish ... View more

Hi,   Can you share the workflow.xml file that you are running?   Regards Nitish ... View more

Hi,   Can you please share the job logs from YARN page? Or you can run the below command.   --> yarn logs -applicationId <application ID>  -appOwner <AppOwner>   This will tell us more info that why the job is failing.   Regards Nitish ... View more

Hi,   We don't see any downside for using that but writing the code for that is totally depend on end user.   Regards Nitish ... View more

Hi,   Generally we recommend to run the Sqoop command from CLI. This is what we support.   Regards Nitish ... View more

Hi,   The probable reason for this job failure is that YARN is not allowing you to submit the job in this queue. Either this queue is not present over there or the placement rules are causing some issue.   There could be a scenario where this user has not been given the permission to submit the job in this queue.   1. Can you share the snapshot of the YARN dynamic resource pools that you have created. 2. Can you also share the snapshot of the YARN placement policies that you have set. 3. The username who is submitting the Sqoop job. 4. Kindly share the snapshot of the users you have given the access to this queue where you are submitting the job.   This will help us to check if there is some issue with the queue or permissions or placement policies.   Regards Nitish ... View more

Hi,   As this is a different issue, I would request to open a new thread because there are multiple issues that has been discussed over here which is making this thread more complicating.   I would request to open a new thread and kindly share the link so we can work on it.   Regards Nitish ... View more

Hi,    you can create Oozie workflow jobs( which will run Sqoop actions frequently) and pass it out in coordinator.   Link:- https://oozie.apache.org/docs/4.1.0/DG_SqoopActionExtension.html   (To create workflow.xml) Link:- https://oozie.apache.org/docs/3.1.3-incubating/CoordinatorFunctionalSpec.html   ( To create coordinator)   Regards Nitish ... View more

Hi,   You are welcome. Regards Nitish ... View more

Hi,   When ever you create sqoop job then it gets created as local metastore under $home/.sqoop folder.   I woul request you to confgiure Global metastore so that all the jobs can be accessed from other nodes as well.   Use the [Best Practices for Sqoop1 Metastore] instructions below to manage the Sqoop1 Metastore   [Best Practices for Sqoop1 Metastore] 1. Identify a cluster host (e.g. metastore1.cloudera.com ) upon which to run the Metastore process 2. Use the [Deploy Sqoop1 Client Gateway] instructions to deploy it on the Metatore host 3. Use the [Configure Sqoop1 Metastore Directory] instructions to create a directory for the Metastore database 4. Use the [Set Sqoop1 Safety Valve sqoop-site.xml ( metastore )] instructions to configure the Metastore 5. Use the [Start Sqoop1 Metastore Java Process] instructions to start the Metastore process 6. Use the [Stop Sqoop1 Metastore Java Process] instructions to gracefully stop the Metastore process     ~~~~~   [Deploy Sqoop1 Client Gateway] 1. Login to Cloudera Manager 2. Navigate to: (Home > Cluster) 3. Click button to the right of ClusterName 4. Select "Add a Service" 5. Select "Sqoop 1 Client" 6. Deploy the Gateway on each host that will run Sqoop1 CLI commands 7. Complete wizard   Also see Cloudera Documentation (Deploy Sqoop Client Gateway): http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/cm_mc_sqoop1_client...   ~~~~~   [Configure Sqoop1 Metastore Directory] 1. Execute [commands] to setup the Sqoop1 Metastore directory   [commands] ssh root@metastore.cloudera.com mkdir /var/lib/sqoop-metastore (can be anywhere) chown sqoop:sqoop /var/lib/sqoop-metastore   ~~~~~   [Set Sqoop1 Safety Valve sqoop-site.xml ( metastore )] 1. Login to Cloudera Manager 2. Navigate to: (Home > Cluster > Sqoop1 Client Gateway > Configuration) 3. Search for "Sqoop 1 Client Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml" 4. Add the [xml properties] below 5. Save Changes 6. Redeploy Sqoop1 Client Gateway   [xml properties] <!-- START SQOOP1 CLIENT GATEWAY SQOOP-SITE.XML SAFETY VALVE CONFIGURATION --> <property> <name>sqoop.metastore.client.autoconnect.url</name> <value>jdbc:hsqldb: hsql://metastore1.cloudera.com:16000/sqoop</value > <description>THIS TELLS SQOOP1 WHERE TO GO TO CONNECT TO SHARED METASTORE</description> </property> <property> <name>sqoop.metastore.server.location</name> <value>/var/lib/sqoop-metastore/metastore.db</value> <description>THIS TELLS SQOOP1 THE LINX PATH AND METASTORE NAME FOR THE DATABASE</description> </property> <property> <name>sqoop.metastore.server.port</name> <value>16000</value> <description>THIS THE LISTEN PORT FOR HSQLDB</description> </property> <property> <name>sqoop.metastore.client.record.password</name> <value>false</value> <description>IF TRUE, SAVING PASSWORDS IN METASTORE IS ALLOWED</description> </property> <property> <name>sqoop.metastore.client.enable.autoconnect</name> <value>true</value> <description>IF TRUE, SQOOP WILL USE LOCAL METASTORE WHEN NO OTHER METASTORE ARGUMENTS ARE PROVIDED </description> </property> <!-- SQOOP1 CLIENT GATEWAY SQOOP-SITE.XML SAFETY VALVE CONFIGURATION -->   ~~~~~   [Start Sqoop1 Metastore Java Process] 1. Execute [commands] to start the Sqoop1 Metastore process   [commands] ssh root@metastore.cloudera.com sudo -u sqoop nohup sqoop-metastore& ps -aux | egrep -i metastore   Output: root      3021  0.1  0.1 169388  3024 pts/1    S    01:49   0:00 sudo -u sqoop sqoop-metastore sqoop     3022 11.8  3.9 1535656 75732 pts/1   Sl   01:49   0:02 /usr/java/jdk1.7.0_67-cloudera/bin/java -Xmx1000m -Dhadoop.log.dir=/opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hadoop/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hadoop -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=/opt/cloudera/parcels/GPLEXTRAS-5.7.1-1.cdh5.7.1.p0.11/lib/hadoop/lib/native:::/opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Dhadoop.security.logger=INFO,NullAppender org.apache.sqoop.Sqoop metastore root      3215  0.0  0.0 101012   860 pts/1    S+   01:49   0:00 egrep -i metastore   ~~~~~   [Stop Sqoop1 Metastore Database] 1. Execute [commands] to stop the Sqoop1 Metastore   [commands] ssh root@metastore.cloudera.com sudo -u sqoop sqoop-metastore --shutdown ps -aux | egrep -i metastore   Hope this helps.     Regards Nitish ... View more

Hi,   You can create Sqoop jobs which will store the value for last value into the embedded database.   Link:- https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_saved_jobs   Kindly go through above link which will help you to achieve this.   Regards Nitish ... View more

Hi,   Can you please trying adding the below configuration in distcp action and see if that works or not.   <property> <name>oozie.launcher.mapreduce.job.hdfs-servers</name> <value>${nameNode1},${nameNode2}</value> </property>   Regards Nitish ... View more

Hi,   This is a bug in Sqoop that it doesn't update the last-value when Hbase is involved. Untill there is a permanent fix, you need to add --target-dir in the sqoop command to make it run fine.   I guess you can put --target-dir to any HDFS path.   Hope this helps.   Regards Nitish ... View more

Hi,   I am afraid if you are passing the shell script name under file tag as well or not and because of that oozie is not able to localize the shell script which is causing the error of "No such file or directory"   Can you please share the worfkflow.xml and job.properties file so we can check if you are passing it in right way or not.   Regards Nitish ... View more

Hi,   Another Link:- https://stackoverflow.com/questions/21076179/pkix-path-building-failed-and-unable-to-find-valid-certification-path-to-requ   Regards Nitish ... View more

Hi,   The reason why you are facing this issue is because the JDK cacerts doesn't have the truststore certificates for your Oozie SSL.   So for your client from where you are running the oozie job, you need to merge the trustore certificates in JDK package.   Instructions:-   Copy or download the root CA cert file (.cert or .pem) onto each client machine * NOTE: Repeat these steps for each client machine. Run the following command to import the certificate into the JRE's keystore. This allows any Java program, including the Oozie client, to connect to the Oozie Server using the certificate. sudo keytool -import -alias tomcat -file path/to/certificate.cert -keystore ${JRE_cacerts} Where   ${JRE_cacerts}   is the path to the JRE's certs file. The file location may differ depending on the Operating System (it is typically called   cacerts   and located at   ${JAVA_HOME}/lib/security/cacerts   but may be under a different directory in   ${JAVA_HOME} Important: Create a backup copy of the   cacerts   file. The default password is   changeit .   After merging this you can run the oozie jobs.   Regards Nitish ... View more

Hi,   Just to be accurate, Kindly check for these 2 jars as per your CDH version and see if there are multiple versions of jar for these 2.   ##  hive-jdbc-1.1.0-cdh5.12.2-standalone.jar matches ## hive-service-1.1.0-cdh5.12.2.jar matches   Regards Nitish ... View more

Hi,   Kindly check if there are some conflicting jars in the stdout logs where you are not able to connect to Hive.   As this is an intermittent failure, I am afraid this could be the cause.   Regards Nitish ... View more

Link:- https://www.tekstream.com/oracle-error-messages/ora-12505-tns-listener-does-not-currently-know-of-sid-given-in-connect-descriptor/ ... View more