Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Oozie workflow with sqoop export failed while using more than 1 mapper in command.

Oozie workflow with sqoop export failed while using more than 1 mapper in command.

New Contributor

Hi Team,

 

We are trying to export data from hive table to RDBMS but while using sqoop export command with more than 1 mapper. Oozie workflow is failing with the following exception :

org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/utils/MetaStoreUtils
at org.apache.hive.hcatalog.mapreduce.InternalUtil.getSerdeProperties(InternalUtil.java:165)
at org.apache.hive.hcatalog.mapreduce.InternalUtil.initializeDeserializer(InternalUtil.java:155)
at org.apache.hive.hcatalog.mapreduce.HCatRecordReader.createDeserializer(HCatRecordReader.java:127)
at org.apache.hive.hcatalog.mapreduce.HCatRecordReader.initialize(HCatRecordReader.java:92)
at org.apache.sqoop.mapreduce.hcat.SqoopHCatRecordReader.initialize(SqoopHCatRecordReader.java:68)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:554)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:792)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.metastore.utils.MetaStoreUtils
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 13 more

 

Can you please assist.

Regards,

Deepa

1 REPLY 1
Highlighted

Re: Oozie workflow with sqoop export failed while using more than 1 mapper in command.

Contributor

@Djain 

Hi Deepa,

 

It seems you are missing the Hive Clients from the NodeManagers, which could be causing the issue.

Can you please make sure you have hive clients installed on all the NodeManager nodes and then confirm you are hitting the issue.

 

Also, if you are facing the problem, will you be able to upload the complete yarn app logs.

 

Thanks,

Rohit Rai Malhotra

Don't have an account?
Coming from Hortonworks? Activate your account here