Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Oozie workflow with sqoop export failed while using more than 1 mapper in command.

New Contributor

Hi Team,

 

We are trying to export data from hive table to RDBMS but while using sqoop export command with more than 1 mapper. Oozie workflow is failing with the following exception :

org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/utils/MetaStoreUtils
at org.apache.hive.hcatalog.mapreduce.InternalUtil.getSerdeProperties(InternalUtil.java:165)
at org.apache.hive.hcatalog.mapreduce.InternalUtil.initializeDeserializer(InternalUtil.java:155)
at org.apache.hive.hcatalog.mapreduce.HCatRecordReader.createDeserializer(HCatRecordReader.java:127)
at org.apache.hive.hcatalog.mapreduce.HCatRecordReader.initialize(HCatRecordReader.java:92)
at org.apache.sqoop.mapreduce.hcat.SqoopHCatRecordReader.initialize(SqoopHCatRecordReader.java:68)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:554)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:792)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.metastore.utils.MetaStoreUtils
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 13 more

 

Can you please assist.

Regards,

Deepa

1 REPLY 1

Contributor

@Djain 

Hi Deepa,

 

It seems you are missing the Hive Clients from the NodeManagers, which could be causing the issue.

Can you please make sure you have hive clients installed on all the NodeManager nodes and then confirm you are hitting the issue.

 

Also, if you are facing the problem, will you be able to upload the complete yarn app logs.

 

Thanks,

Rohit Rai Malhotra