Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Sqoop hive import through oozie

Solved Go to solution

Sqoop hive import through oozie

Explorer

I have a oozie workflow which launched Sqoop Job against Hive, While importing it is throwing the below error,

[main] ERROR org.apache.sqoop.tool.ImportTool - Encountered IOException running import job: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf

at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:50) at org.apache.sqoop.hive.HiveImport.getHiveArgs(HiveImport.java:397) at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:384) at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:342) at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:246) at org.apache.sqoop.tool.CreateHiveTableTool.run(CreateHiveTableTool.java:58) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:225) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.main(Sqoop.java:243) at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:202) at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:182) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:51) at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:48) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Can someone help me out on this?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Sqoop hive import through oozie

@arjun more

You can follow below steps to get it resolved,

- Add the following classpath in file "/etc/sqoop/conf/sqoop-env.sh"

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/hdp/current/hive-client/lib/*

- Make sure you have installed the Hive client in the node where Oozie server is running.

- Add hive-exec jar in oozie lib

- Also make sure yarn user is present in HDFS(/user/yarn directory).

And restart Oozie server and try again.

View solution in original post

6 REPLIES 6
Highlighted

Re: Sqoop hive import through oozie

@arjun more

You can follow below steps to get it resolved,

- Add the following classpath in file "/etc/sqoop/conf/sqoop-env.sh"

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/hdp/current/hive-client/lib/*

- Make sure you have installed the Hive client in the node where Oozie server is running.

- Add hive-exec jar in oozie lib

- Also make sure yarn user is present in HDFS(/user/yarn directory).

And restart Oozie server and try again.

View solution in original post

Highlighted

Re: Sqoop hive import through oozie

Explorer

@nshelke

I tried your suggestion, only not sure how to add user in HDFS. Can you guide me?

Highlighted

Re: Sqoop hive import through oozie

Explorer

Its giving me error while running the job that, Permission denied for user yarn. Not sure why as I am running this oozie action by arjun user still getting this error.

Highlighted

Re: Sqoop hive import through oozie

@arjun more

You can use below commands for adding user directory in hdfs,

su - <HDFS_USER>

hdfs dfs -mkdir /user/yarn

hdfs dfs -chown -R yarn:hdfs /user/yarn

This will create the user in HDFS.

Highlighted

Re: Sqoop hive import through oozie

Explorer
@nshelke

Thanks, it really helped. And now oozie job is running.

Highlighted

Re: Sqoop hive import through oozie

Expert Contributor

Why add yarn user directory in hdfs?

Don't have an account?
Coming from Hortonworks? Activate your account here