Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

oozie workflow failed with sqoop and hive import

Hello all,

I am new to oozie. I have oozie wf working successful with sqoop command but when I added hive import into oozie workflow I got it failed as "Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]"

Can anyone please help me? I really appreciated. Thanks

Here is my workflow:

/************

<?xml version="1.0" encoding="UTF-8"?>

<workflow-app xmlns="uri:oozie:workflow:0.2" name="sqoop-wf">

<start to="sqoop-node"/> <action name="sqoop-node">

<sqoop xmlns="uri:oozie:sqoop-action:0.2">

<job-tracker>${jobTracker}</job-tracker>

<name-node>${nameNode}</name-node>

<prepare>

<delete path="test-oozie/examples/output-data/sqoop"/>

<mkdir path="test-oozie/examples/output-data"/> </prepare>

<configuration>

<property> <name>mapred.job.queue.name</name>

<value>${queueName}</value> </property>

<property> <name>oozie.hive.defaults</name>

<value>/examples/apps/sqoop/hive-site.xml</value> </property> </configuration>

<arg>import</arg> <arg>--connect</arg> <arg>jdbc:oracle:thin:@hostname</arg> <arg>--username</arg> <arg>usename</arg> <arg>--password</arg> <arg>pwd</arg> <arg>--query</arg> <arg><![CDATA[SELECT ID, STARTTIME FROM TEST WHERE STARTTIME BETWEEN TO_DATE('2017-05-30 13:00:00', 'YYYY-MM-DD HH24:MI:SS') AND TO_DATE('2017-05-30 13:15:59','YYYY-MM-DD HH24:MI:SS') AND (1=1 OR $CONDITIONS)]]></arg>

<arg>--hive-import</arg> <arg>--hive-drop-import-delims</arg> <arg>--hive-overwrite</arg> <arg>--hive-table</arg> <arg>task_detailstest_tmp</arg> <arg>--target-dir</arg> <arg>/apps/hive/warehouse/task_detailstest</arg> <arg>-m</arg> <arg>1</arg>

</sqoop> <ok to="end"/>

<error to="fail"/>

</action> <kill name="fail"> <message>Sqoop failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app>

4 REPLIES 4

@JT Ng

I suspect this can be issue with missing hive-site and tez-site.

You can find more hint in yarn MR job log launched by oozie.

Hello, I have these file in my workflow now

<file>/test-oozie/examples/apps/sqoop/hive-site.xml</file>

<file>/test-oozie/examples/apps/sqoop/tez.xml</file>

But the errror now is "JA008: File does not exist hdfs://localhost/test-oozie/examples/apps/sqoop/tez.xml#tez.xml"

Even though I copied the Tez jars and tez.tar.gz into hdfs. Which jar am I missing?

Thanks

Explorer

Why HDFS port is missing in the error or it is a HDFS HA with nameservice?

Rising Star

@JT Ng

>> But the errror now is "JA008: File does not exist hdfs://localhost/test-oozie/examples/apps/sqoop/tez.xml#tez.xml"

Did you mean to specify tez-site.xml?