Created 04-29-2016 02:03 PM
workflow.xml:
<workflow-app name="once-a-day" xmlns="uri:oozie:workflow:0.1"> <start to="sqoopAction"/> <action name="sqoopAction"> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <command>import-all-tables --connect jdbc:mysql://xyz.syz/erp --username hive --password hive --export-dir /user/hive/warehouse/sbl </command> </sqoop> <ok to="end"/> <error to="killJob"/> </action> <kill name="killJob"> <message>"Killed job due to error: ${wf:errorMessage(wf:lastErrorNode())}"</message> </kill> <end name="end" /> </workflow-app>
job.properties:
nameNode=hdfs://syz.syz.com:8020 jobTracker=xyz.syz.com:8050 queueName=default oozie.use.system.libpath=true oozie.coord.application.path=${nameNode}/user/${user.name}/scheduledimport start=2013-09-01T00:00Z end=2013-12-31T00:00Z workflowAppUri=${nameNode}/user/${user.name}/scheduledimport
I get an error on sqoop task:
java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.SqoopMain not found
I have share/lib inside /user/oozie.
How do I fix this?
Created 05-06-2016 07:18 PM
Please list the content of your shared lib:
hdfs dfs -ls /user/oozie/share/lib/lib_{20160430195631}
hdfs dfs -ls -R /user/oozie/share/lib/lib_{20160430195631}/oozie
I can check with my installation if there is any missing jar.
Created 04-29-2016 07:47 PM
Does your oozie shared lib in hdfs have sqoop related dependencies?
If not set up sharedlib following this link
Created 05-02-2016 09:57 AM
Yes, I sure do. What else could possibly be the problem?
Created 05-02-2016 11:47 AM
Hi @simran kaur,
You can list the available sharelibs with the following command:
sudo -u oozie oozie admin -shareliblist -oozie http://<oozie_host>:11000/oozie [Available ShareLib] oozie hive distcp hcatalog sqoop mapreduce-streaming spark hive2 pig
Also, is the mysql driver in your lib folder of the workflow application as described here? https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a7_Workflow_Application_Deployment
Hope this helps, Chris
Created 05-02-2016 12:05 PM
This is what I have in my sharelib:
-shareliblist [Available ShareLib]
hive mapreduce-streaming oozie sqoop pig
Still I keep getting the same error. Yes, I do have the lib folder along with the connector jar
Created 05-02-2016 12:25 PM
Created 05-02-2016 12:40 PM
Does this work?
sqoop list-databases --connect jdbc:mysql://xyz.syz/ --username hive --password hive
Created 05-02-2016 01:33 PM
@Christian Guegi : No, I still get the same error. Not even list databases or table works.
Created 05-02-2016 01:36 PM
@Christian Guegi: I could run import and list commands from the command line directly thought.Somehow oozie is not able to pick up the share/lib. Also, is sharelib different from share/lib? I have share/lib in /user/oozie but nothing called sharelib.
Created 05-05-2016 12:10 PM
@Christian Guegi: Now, I did a fresh install and can run list-databases and list-tables command but not import. I do have jdbc driver inside of lib folder in same directory as that of workflow.xml but do I need to reference to this jar inside lib folder from any of my files?