Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Sqoop hive import through oozie

avatar
Rising Star

I have a oozie workflow which launched Sqoop Job against Hive, While importing it is throwing the below error,

[main] ERROR org.apache.sqoop.tool.ImportTool - Encountered IOException running import job: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf

at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:50) at org.apache.sqoop.hive.HiveImport.getHiveArgs(HiveImport.java:397) at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:384) at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:342) at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:246) at org.apache.sqoop.tool.CreateHiveTableTool.run(CreateHiveTableTool.java:58) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:225) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.main(Sqoop.java:243) at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:202) at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:182) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:51) at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:48) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Can someone help me out on this?

1 ACCEPTED SOLUTION

avatar
@arjun more

You can follow below steps to get it resolved,

- Add the following classpath in file "/etc/sqoop/conf/sqoop-env.sh"

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/hdp/current/hive-client/lib/*

- Make sure you have installed the Hive client in the node where Oozie server is running.

- Add hive-exec jar in oozie lib

- Also make sure yarn user is present in HDFS(/user/yarn directory).

And restart Oozie server and try again.

View solution in original post

6 REPLIES 6

avatar
@arjun more

You can follow below steps to get it resolved,

- Add the following classpath in file "/etc/sqoop/conf/sqoop-env.sh"

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/hdp/current/hive-client/lib/*

- Make sure you have installed the Hive client in the node where Oozie server is running.

- Add hive-exec jar in oozie lib

- Also make sure yarn user is present in HDFS(/user/yarn directory).

And restart Oozie server and try again.

avatar
Rising Star

@nshelke

I tried your suggestion, only not sure how to add user in HDFS. Can you guide me?

avatar
Rising Star

Its giving me error while running the job that, Permission denied for user yarn. Not sure why as I am running this oozie action by arjun user still getting this error.

avatar

@arjun more

You can use below commands for adding user directory in hdfs,

su - <HDFS_USER>

hdfs dfs -mkdir /user/yarn

hdfs dfs -chown -R yarn:hdfs /user/yarn

This will create the user in HDFS.

avatar
Rising Star
@nshelke

Thanks, it really helped. And now oozie job is running.

avatar
Expert Contributor

Why add yarn user directory in hdfs?