Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 04-29-2016 02:03 PM
workflow.xml:
<workflow-app name="once-a-day" xmlns="uri:oozie:workflow:0.1">
<start to="sqoopAction"/>
<action name="sqoopAction">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<command>import-all-tables --connect jdbc:mysql://xyz.syz/erp --username hive --password hive
--export-dir /user/hive/warehouse/sbl
</command>
</sqoop>
<ok to="end"/>
<error to="killJob"/>
</action>
<kill name="killJob">
<message>"Killed job due to error: ${wf:errorMessage(wf:lastErrorNode())}"</message>
</kill>
<end name="end" />
</workflow-app>job.properties:
nameNode=hdfs://syz.syz.com:8020
jobTracker=xyz.syz.com:8050
queueName=default
oozie.use.system.libpath=true
oozie.coord.application.path=${nameNode}/user/${user.name}/scheduledimport
start=2013-09-01T00:00Z
end=2013-12-31T00:00Z
workflowAppUri=${nameNode}/user/${user.name}/scheduledimportI get an error on sqoop task:
java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.SqoopMain not found
I have share/lib inside /user/oozie.
How do I fix this?
Created 05-06-2016 07:18 PM
Please list the content of your shared lib:
hdfs dfs -ls /user/oozie/share/lib/lib_{20160430195631}
hdfs dfs -ls -R /user/oozie/share/lib/lib_{20160430195631}/oozie
I can check with my installation if there is any missing jar.
Created 04-29-2016 07:47 PM
Does your oozie shared lib in hdfs have sqoop related dependencies?
If not set up sharedlib following this link
Created 05-02-2016 09:57 AM
Yes, I sure do. What else could possibly be the problem?
Created 05-02-2016 11:47 AM
Hi @simran kaur,
You can list the available sharelibs with the following command:
sudo -u oozie oozie admin -shareliblist -oozie http://<oozie_host>:11000/oozie [Available ShareLib] oozie hive distcp hcatalog sqoop mapreduce-streaming spark hive2 pig
Also, is the mysql driver in your lib folder of the workflow application as described here? https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a7_Workflow_Application_Deployment
Hope this helps, Chris
Created 05-02-2016 12:05 PM
This is what I have in my sharelib:
-shareliblist [Available ShareLib]
hive mapreduce-streaming oozie sqoop pig
Still I keep getting the same error. Yes, I do have the lib folder along with the connector jar
Created 05-02-2016 12:25 PM
Created 05-02-2016 12:40 PM
Does this work?
sqoop list-databases --connect jdbc:mysql://xyz.syz/ --username hive --password hive
Created 05-02-2016 01:33 PM
@Christian Guegi : No, I still get the same error. Not even list databases or table works.
Created 05-02-2016 01:36 PM
@Christian Guegi: I could run import and list commands from the command line directly thought.Somehow oozie is not able to pick up the share/lib. Also, is sharelib different from share/lib? I have share/lib in /user/oozie but nothing called sharelib.
Created 05-05-2016 12:10 PM
@Christian Guegi: Now, I did a fresh install and can run list-databases and list-tables command but not import. I do have jdbc driver inside of lib folder in same directory as that of workflow.xml but do I need to reference to this jar inside lib folder from any of my files?