Support Questions
Find answers, ask questions, and share your expertise

Sqoop hive import through oozie

Explorer

I have a oozie workflow which launched Sqoop Job against Hive, While importing it is throwing the below error,

[main] ERROR org.apache.sqoop.tool.ImportTool - Encountered IOException running import job: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf

at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:50) at org.apache.sqoop.hive.HiveImport.getHiveArgs(HiveImport.java:397) at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:384) at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:342) at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:246) at org.apache.sqoop.tool.CreateHiveTableTool.run(CreateHiveTableTool.java:58) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:225) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.main(Sqoop.java:243) at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:202) at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:182) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:51) at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:48) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Can someone help me out on this?

1 ACCEPTED SOLUTION

Accepted Solutions

@arjun more

You can follow below steps to get it resolved,

- Add the following classpath in file "/etc/sqoop/conf/sqoop-env.sh"

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/hdp/current/hive-client/lib/*

- Make sure you have installed the Hive client in the node where Oozie server is running.

- Add hive-exec jar in oozie lib

- Also make sure yarn user is present in HDFS(/user/yarn directory).

And restart Oozie server and try again.

View solution in original post

6 REPLIES 6

@arjun more

You can follow below steps to get it resolved,

- Add the following classpath in file "/etc/sqoop/conf/sqoop-env.sh"

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/hdp/current/hive-client/lib/*

- Make sure you have installed the Hive client in the node where Oozie server is running.

- Add hive-exec jar in oozie lib

- Also make sure yarn user is present in HDFS(/user/yarn directory).

And restart Oozie server and try again.

View solution in original post

Explorer

@nshelke

I tried your suggestion, only not sure how to add user in HDFS. Can you guide me?

Explorer

Its giving me error while running the job that, Permission denied for user yarn. Not sure why as I am running this oozie action by arjun user still getting this error.

@arjun more

You can use below commands for adding user directory in hdfs,

su - <HDFS_USER>

hdfs dfs -mkdir /user/yarn

hdfs dfs -chown -R yarn:hdfs /user/yarn

This will create the user in HDFS.

Explorer
@nshelke

Thanks, it really helped. And now oozie job is running.

Expert Contributor

Why add yarn user directory in hdfs?