Created on 02-22-2018 08:54 AM - edited 08-17-2019 05:52 PM
I've create an Oozie Job that uses Sqoop import get data from mysql database to hive.
the strange thing is that on Hadoop job list all my job status are SUCCEEDED instead on Oozie that job is FAIL.
This is my workflow.xml configuration
<?xml version="1.0" encoding="UTF-8"?> <workflow-app xmlns="uri:oozie:workflow:0.2" name="sqoop-importAssurance"> <start to="sqoop-node"/> <action name="sqoop-node"> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <configuration> <property> <name>mapred.job.queue.name</name> <value>${queueName}</value> </property> <property> <name>oozie.hive.defaults</name> <value>/tmp/hive-site.xml</value> </property> <property> <name>yarn.nodemanager.vmem-check-enabled</name> <value>false</value> <description>Whether virtual memory limits will be enforced for containers</description> </property> <property> <name>yarn.nodemanager.vmem-pmem-ratio</name> <value>4</value> <description>Ratio between virtual memory to physical memory when setting memory limits for containers</description> </property> </configuration> <command>import -Dorg.apache.sqoop.splitter.allow_text_splitter=true --connect jdbc:mysql://xxxxxx --table assurance --username xxxxx --password xxxxxx --driver com.mysql.jdbc.Driver --hive-import --hive-table assurance --check-column lastaccess --incremental lastmodified --map-column-hive firstaccess=TIMESTAMP,lastaccess=TIMESTAMP -m 1 --merge-key macaddr</command> <file>/tmp/hive-site.xml#hive-site.xml</file> </sqoop> <ok to="end"/> <error to="fail"/> </action> <kill name="fail"> <message>Sqoop failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app>
and this is myproperties
nameNode=hdfs://172.17.0.2:8020 jobTracker=sandbox-hdp.hortonworks.com:8032 queueName=default examplesRoot=examples oozie.use.system.libpath=true oozie.action.sharelib.for.sqoop=hive,hcatalog,sqoop oozie.action.sharelib.for.hive=hive,hcatalog,sqoop oozie.wf.application.path=${nameNode}/user/test/ workflowAppUri=${nameNode}/user/test/ oozie.libpat=/user/oozie/share/lib/ user.name=root start=2018-02-14T17:30Z end=2018-02-14T19:30Z
I've insert mysql-connector-java-5.1.45 into oozie.libpath folder and uplod hive-conf.xml into /etc/hive/conf/,
but nothing.
below there are my full log, i've set oozie log level to DEBUG
Any suggestion?
thanks in advance
Created 02-22-2018 10:00 PM
@Salvatore Matino Where exactly have you copied the JAR ? Also, can you check which JAR is being picked by looking at the classpath by sqoop service ?
Created 02-23-2018 08:56 AM
Hi @Shyam Sunder Rai i've copied JAR into two folders : /user/oozie/share/lib/lib_20171110144231/oozie and
/user/oozie/share/lib/lib_20171110144231/sqoop
From sqoop classpath when i run it seams that sqoop using my JAR. i've attach my STDOUT logs
thanks a lot
Created 02-28-2018 03:08 PM
Hi @Shyam Sunder Rai any suggestion?
Created 04-16-2018 11:03 AM
Hi Salvator,
Im facing same problem, do u find any solution for this?