Member since
01-13-2016
3
Posts
0
Kudos Received
0
Solutions
01-14-2016
02:34 AM
Thanks. Currently I am using ssh action to call sqoop import as workaround.
... View more
01-14-2016
02:32 AM
thanks this can be another working workaround solution.
... View more
01-13-2016
06:45 AM
I am using HDP 2.3.2, with sqoop 1.4.6, oozie 4.2.0, hive 1.2.1, hadoop 2.7.1. hive warehouse dir: /apps/hive/warehouse I run the following sqoop commands and submit oozie job with user hdfs:hdfs Manually run sqoop import is OK, and it did succeeded imported mysql table to hive table. sqoop import --connect jdbc:mysql://data001.qc.com/demo27 --driver com.mysql.jdbc.Driver --username root --password root --table clb_usertoken --warehouse-dir /user/hive/warehouse --hive-import --hive-overwrite Use oozie workflow to do the sqoop import failed. One of the action always get this error "Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]" And its corresponding mapreduce job is actually SUCCEEDED. I can also find the successfully processed files under /user/hive/warehouse [hdfs@data006 import_from_mysql]$ hdfs dfs -ls -R /user/hive/warehouse drwxr-xr-x - hdfs hdfs 0 2016-01-13 14:05 /user/hive/warehouse/clb_usertoken -rw-r--r-- 3 hdfs hdfs 0 2016-01-13 14:05 /user/hive/warehouse/clb_usertoken/_SUCCESS -rw-r--r-- 3 hdfs hdfs 375228 2016-01-13 14:05 /user/hive/warehouse/clb_usertoken/part-m-00000 -rw-r--r-- 3 hdfs hdfs 372955 2016-01-13 14:05 /user/hive/warehouse/clb_usertoken/part-m-00001 -rw-r--r-- 3 hdfs hdfs 380625 2016-01-13 14:05 /user/hive/warehouse/clb_usertoken/part-m-00002 -rw-r--r-- 3 hdfs hdfs 387143 2016-01-13 14:05 /user/hive/warehouse/clb_usertoken/part-m-00003
So it seems that the final step to import the processed result directory to hive warehouse failed. But I can't find any useful log from oozie.log. Please, can anyone point out what's wrong here? ------------ Attached workflow.xml (2nd action, I've deleted the unnecessary lines after.) & job.properties
security_enabled=False nameNode=hdfs://data002.qc.com:8020 jobTracker=data002.qc.com:8050
hive2JdbcUrl=jdbc:hive2://data005.qc.com:10000/default appRootDir=/qc/app/user_behavior_analysis oozie.wf.application.path=${nameNode}/qc/app/user_behavior_analysis/oozie/import_from_mysql/workflow.xml END.
... View more
Labels: