Reply
Explorer
Posts: 19
Registered: ‎07-19-2016

Sqoop1 jobs fail in kerberized cluster

Hi,

 

Since I enabled kerberos on my cluster, all sqoop jobs that I have scheduled with oozie fails. The sqoop jobs pull tables from mysql and load them into Hive.

 

In Hue I can see that the sqoop job creates mapreduce task and temp files are put on hdfs in /user/myrunning_user/pulled_table

 

Seems like the loading into Hive is failing. The only error I see in Hue related to the failed job is:

 

Logging initialized using configuration in jar:file:/var/yarn/nm/filecache/84/hive-exec.jar!/hive-log4j.properties
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.transport.TTransportException
Intercepting System.exit(1)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]
Highlighted
Posts: 445
Topics: 1
Kudos: 58
Solutions: 36
Registered: ‎08-16-2016

Re: Sqoop1 jobs fail in kerberized cluster

Try to find a failed mapper and get the full exception or other info.  There just isn't enough information here.

 

Is the user in question able to load the data into Hive outside of Sqoop?  Can you run the sqoop job outside of Hue/Oozie?

 

 

Announcements