question Re: Sqoop Oozie Coordinator job killed issue in Support Questions

Sqoop Oozie Coordinator job killed issue

Nekkanti — Thu, 08 Aug 2019 20:59:09 GMT

I am trying to export data from Postgres to Hive through Sqoop in Oozie Coordinator Jobs. I set the frequency to 5 MINS.
The job is started after the start time and the data is loaded into hive for the first time and after 5 mins frequency the job is started and it is killed.
What is the problem here ?
Below are my properties and xml file:

Coordinator.xml file:

<coordinator-app name="coordinator1" frequency="${frequency}" start="${startTime}" end="${endTime}"
timezone="${timezone}" xmlns="uri:oozie:coordinator:0.4">
<action>
<workflow>
<app-path>${workflowPath}</app-path>
</workflow>
</action>
</coordinator-app>

Job.properties:

frequency=5
startTime=2019-08-07T10:22Z
endTime=2019-08-07T10:32Z
timezone=UTC
nameNode=hdfs://HadoopNode3:8020
jobTracker=HadoopNode3:8032
queueName=default
oozie.action.sharelib.for.sqoop=sqoop,hive
oozie.use.system.libpath=true
workflowPath=/user/oozieschedules/sqoop/coordinator
oozie.coord.application.path=/user/oozieschedules/sqoop/coordinator

Workflow.xml:

<workflow-app name="sqoop-coordwf" xmlns="uri:oozie:workflow:0.4">
<start to="sqoop-coordaction"/>
<action name="sqoop-coordaction">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
<delete path="hdfs://HadoopNode3:8020/user/cloudera/student"/>
</prepare>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<arg>import</arg>
<arg>--connect</arg>
<arg>jdbc:postgresql://ipaddress:5432/inet</arg>
<arg>--username</arg>
<arg>postgres</arg>
<arg>--password</arg>
<arg>postgres</arg>
<arg>--table</arg>
<arg>student</arg>
<arg>--hive-import</arg>
<arg>--hive-table</arg>
<arg>test</arg>
<arg>--create-hive-table</arg>
<arg>-m</arg>
<arg> 1</arg>
<file>/user/oozieschedules/sqoop/coordinator/hive-site.xml#hive-site.xml</file>

</sqoop>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Failed, Error Message</message>
</kill>
<end name="end"/>
</workflow-app>

could anyone please resolve the issue .....

Re: Sqoop Oozie Coordinator job killed issue

EricL — Fri, 09 Aug 2019 08:42:21 GMT

Hi,

Without the actual error message, it would be quite hard to troubleshoot issues. For future reference, please share the error log if possible, it helps to determine the cause.

If you want me to guess, I would think it might be caused by the fact that you create same table every time when workflow is triggered. Do you intend to overwrite the table or append data to the table?

Cheers

Eric

Re: Sqoop Oozie Coordinator job killed issue

Nekkanti — Fri, 09 Aug 2019 08:49:04 GMT

I want to append the data to the same table.

Log Error:

Intercepting System.exit(1)

<<< Invocation of Main class completed <<<

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]

Oozie Launcher failed, finishing Hadoop job gracefully.

The log file contains the above error only.

Please help me in resolving the issue..

Re: Sqoop Oozie Coordinator job killed issue

Nekkanti — Fri, 09 Aug 2019 09:38:33 GMT

Thanks a lot.

I removed the create hive table statement.The job is successfully executed now.