Created 06-04-2017 02:18 AM
Hello all,
I am new to oozie. I have oozie wf working successful with sqoop command but when I added hive import into oozie workflow I got it failed as "Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]"
Can anyone please help me? I really appreciated. Thanks
Here is my workflow:
/************
<?xml version="1.0" encoding="UTF-8"?>
<workflow-app xmlns="uri:oozie:workflow:0.2" name="sqoop-wf">
<start to="sqoop-node"/> <action name="sqoop-node">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
<delete path="test-oozie/examples/output-data/sqoop"/>
<mkdir path="test-oozie/examples/output-data"/> </prepare>
<configuration>
<property> <name>mapred.job.queue.name</name>
<value>${queueName}</value> </property>
<property> <name>oozie.hive.defaults</name>
<value>/examples/apps/sqoop/hive-site.xml</value> </property> </configuration>
<arg>import</arg> <arg>--connect</arg> <arg>jdbc:oracle:thin:@hostname</arg> <arg>--username</arg> <arg>usename</arg> <arg>--password</arg> <arg>pwd</arg> <arg>--query</arg> <arg><![CDATA[SELECT ID, STARTTIME FROM TEST WHERE STARTTIME BETWEEN TO_DATE('2017-05-30 13:00:00', 'YYYY-MM-DD HH24:MI:SS') AND TO_DATE('2017-05-30 13:15:59','YYYY-MM-DD HH24:MI:SS') AND (1=1 OR $CONDITIONS)]]></arg>
<arg>--hive-import</arg> <arg>--hive-drop-import-delims</arg> <arg>--hive-overwrite</arg> <arg>--hive-table</arg> <arg>task_detailstest_tmp</arg> <arg>--target-dir</arg> <arg>/apps/hive/warehouse/task_detailstest</arg> <arg>-m</arg> <arg>1</arg>
</sqoop> <ok to="end"/>
<error to="fail"/>
</action> <kill name="fail"> <message>Sqoop failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app>
Created 06-04-2017 06:48 AM
I suspect this can be issue with missing hive-site and tez-site.
You can find more hint in yarn MR job log launched by oozie.
Created 06-04-2017 01:38 PM
Hello, I have these file in my workflow now
<file>/test-oozie/examples/apps/sqoop/hive-site.xml</file>
<file>/test-oozie/examples/apps/sqoop/tez.xml</file>
But the errror now is "JA008: File does not exist hdfs://localhost/test-oozie/examples/apps/sqoop/tez.xml#tez.xml"
Even though I copied the Tez jars and tez.tar.gz into hdfs. Which jar am I missing?
Thanks
Created 06-07-2017 08:37 AM
Why HDFS port is missing in the error or it is a HDFS HA with nameservice?
Created 06-06-2017 06:26 PM
>> But the errror now is "JA008: File does not exist hdfs://localhost/test-oozie/examples/apps/sqoop/tez.xml#tez.xml"
Did you mean to specify tez-site.xml?