Support Questions
Find answers, ask questions, and share your expertise

oozie workflow failed with sqoop and hive import

Hello all,

I am new to oozie. I have oozie wf working successful with sqoop command but when I added hive import into oozie workflow I got it failed as "Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]"

Can anyone please help me? I really appreciated. Thanks

Here is my workflow:

/************

<?xml version="1.0" encoding="UTF-8"?>

<workflow-app xmlns="uri:oozie:workflow:0.2" name="sqoop-wf">

<start to="sqoop-node"/> <action name="sqoop-node">

<sqoop xmlns="uri:oozie:sqoop-action:0.2">

<job-tracker>${jobTracker}</job-tracker>

<name-node>${nameNode}</name-node>

<prepare>

<delete path="test-oozie/examples/output-data/sqoop"/>

<mkdir path="test-oozie/examples/output-data"/> </prepare>

<configuration>

<property> <name>mapred.job.queue.name</name>

<value>${queueName}</value> </property>

<property> <name>oozie.hive.defaults</name>

<value>/examples/apps/sqoop/hive-site.xml</value> </property> </configuration>

<arg>import</arg> <arg>--connect</arg> <arg>jdbc:oracle:thin:@hostname</arg> <arg>--username</arg> <arg>usename</arg> <arg>--password</arg> <arg>pwd</arg> <arg>--query</arg> <arg><![CDATA[SELECT ID, STARTTIME FROM TEST WHERE STARTTIME BETWEEN TO_DATE('2017-05-30 13:00:00', 'YYYY-MM-DD HH24:MI:SS') AND TO_DATE('2017-05-30 13:15:59','YYYY-MM-DD HH24:MI:SS') AND (1=1 OR $CONDITIONS)]]></arg>

<arg>--hive-import</arg> <arg>--hive-drop-import-delims</arg> <arg>--hive-overwrite</arg> <arg>--hive-table</arg> <arg>task_detailstest_tmp</arg> <arg>--target-dir</arg> <arg>/apps/hive/warehouse/task_detailstest</arg> <arg>-m</arg> <arg>1</arg>

</sqoop> <ok to="end"/>

<error to="fail"/>

</action> <kill name="fail"> <message>Sqoop failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app>

4 REPLIES 4

Re: oozie workflow failed with sqoop and hive import

@JT Ng

I suspect this can be issue with missing hive-site and tez-site.

You can find more hint in yarn MR job log launched by oozie.

Re: oozie workflow failed with sqoop and hive import

Hello, I have these file in my workflow now

<file>/test-oozie/examples/apps/sqoop/hive-site.xml</file>

<file>/test-oozie/examples/apps/sqoop/tez.xml</file>

But the errror now is "JA008: File does not exist hdfs://localhost/test-oozie/examples/apps/sqoop/tez.xml#tez.xml"

Even though I copied the Tez jars and tez.tar.gz into hdfs. Which jar am I missing?

Thanks

Re: oozie workflow failed with sqoop and hive import

Explorer

Why HDFS port is missing in the error or it is a HDFS HA with nameservice?

Re: oozie workflow failed with sqoop and hive import

Rising Star

@JT Ng

>> But the errror now is "JA008: File does not exist hdfs://localhost/test-oozie/examples/apps/sqoop/tez.xml#tez.xml"

Did you mean to specify tez-site.xml?