I am unable to run the sqoop jobs on the oozie scheduler. it is an incremental import sqoop job so we need to to schedule this as a sqoop job but having issues on it. But normal sqoop commands are running on the oozie scheduler.
The sqoop job works perfectly when run it on the server but we want to schedule the same job in Oozie. But when we run it on Oozie it throws the below error and the job fails.
"ERROR, reason: Main Class [org.apache.oozie.action.hadoop.SqoopMain], exit code "
when we run a jar file or sqoop command on oozie, it works fine. only the sqoop job fails with the error. we need to schedule several import sqoop jobs in our cluster , so we need to fix this asap.
I later added the jar file on the sqoop hdfs path hdfs dfs -put log4j-jcl-2.13.3.jar /user/oozie/share/lib/lib/sqoop After adding this file, it gives a different error. it says the job is not found..
118:16:17.086 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration. 18:16:17.119 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version: 184.108.40.206.1.4.0-203 18:16:17.170 [main] INFO org.apache.sqoop.manager.SqlManager - Using default fetchSize of 1000 18:16:17.385 [main] ERROR org.apache.sqoop.metastore.GenericJobStorage - Cannot restore job: AppRiskMASRiskCategories1 18:16:17.385 [main] ERROR org.apache.sqoop.metastore.GenericJobStorage - (No such job) 18:16:17.386 [main] ERROR org.apache.sqoop.tool.JobTool - I/O error performing job operation: java.io.IOException: Cannot restore missing job AppRiskMASRiskCategories1 at org.apache.sqoop.metastore.GenericJobStorage.read(GenericJobStorage.java:272)
I also tried to create a sqoop job on the server where the yarn is running. but that also gives the same issue error.
I need to confirm if it is sqoop jobs are specific to the users because we have a user with which we login to the Hue and schedule the sqoop job on the hue console but the sqoop job on the server is with a different user. May be that's why it is not able to locate the sqoop job.
I want to check what are the guidelines to run/schedule a sqoop job on hue.
1. related to users
2. on which server the sqoop jobs needs to be placed
3. what are the other dependencies.
Please help on how to run a sqoop job on oozie scheduler . Thanks a lot