Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

oozie workflow failed with sqoop and hive import

oozie workflow failed with sqoop and hive import

New Contributor

Hello all,

I am new to oozie. I have oozie wf working successful with sqoop command but when I added hive import into oozie workflow I got it failed as "Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]"

Can anyone please help me? I really appreciated. Thanks

Here is my workflow:

/************

<?xml version="1.0" encoding="UTF-8"?>

<workflow-app xmlns="uri:oozie:workflow:0.2" name="sqoop-wf">

<start to="sqoop-node"/> <action name="sqoop-node">

<sqoop xmlns="uri:oozie:sqoop-action:0.2">

<job-tracker>${jobTracker}</job-tracker>

<name-node>${nameNode}</name-node>

<prepare>

<delete path="test-oozie/examples/output-data/sqoop"/>

<mkdir path="test-oozie/examples/output-data"/> </prepare>

<configuration>

<property> <name>mapred.job.queue.name</name>

<value>${queueName}</value> </property>

<property> <name>oozie.hive.defaults</name>

<value>/examples/apps/sqoop/hive-site.xml</value> </property> </configuration>

<arg>import</arg> <arg>--connect</arg> <arg>jdbc:oracle:thin:@hostname</arg> <arg>--username</arg> <arg>usename</arg> <arg>--password</arg> <arg>pwd</arg> <arg>--query</arg> <arg><![CDATA[SELECT ID, STARTTIME FROM TEST WHERE STARTTIME BETWEEN TO_DATE('2017-05-30 13:00:00', 'YYYY-MM-DD HH24:MI:SS') AND TO_DATE('2017-05-30 13:15:59','YYYY-MM-DD HH24:MI:SS') AND (1=1 OR $CONDITIONS)]]></arg>

<arg>--hive-import</arg> <arg>--hive-drop-import-delims</arg> <arg>--hive-overwrite</arg> <arg>--hive-table</arg> <arg>task_detailstest_tmp</arg> <arg>--target-dir</arg> <arg>/apps/hive/warehouse/task_detailstest</arg> <arg>-m</arg> <arg>1</arg>

</sqoop> <ok to="end"/>

<error to="fail"/>

</action> <kill name="fail"> <message>Sqoop failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app>

4 REPLIES 4

Re: oozie workflow failed with sqoop and hive import

@JT Ng

I suspect this can be issue with missing hive-site and tez-site.

You can find more hint in yarn MR job log launched by oozie.

Re: oozie workflow failed with sqoop and hive import

New Contributor

Hello, I have these file in my workflow now

<file>/test-oozie/examples/apps/sqoop/hive-site.xml</file>

<file>/test-oozie/examples/apps/sqoop/tez.xml</file>

But the errror now is "JA008: File does not exist hdfs://localhost/test-oozie/examples/apps/sqoop/tez.xml#tez.xml"

Even though I copied the Tez jars and tez.tar.gz into hdfs. Which jar am I missing?

Thanks

Re: oozie workflow failed with sqoop and hive import

New Contributor

Why HDFS port is missing in the error or it is a HDFS HA with nameservice?

Re: oozie workflow failed with sqoop and hive import

Rising Star

@JT Ng

>> But the errror now is "JA008: File does not exist hdfs://localhost/test-oozie/examples/apps/sqoop/tez.xml#tez.xml"

Did you mean to specify tez-site.xml?

Don't have an account?
Coming from Hortonworks? Activate your account here