Support Questions
Find answers, ask questions, and share your expertise

Error when I'm trying to call sqoop 1.4.6 in HDP 2.5.3 through Java API, Sqoop.runTool, using HCATALOG

Error when I'm trying to call sqoop 1.4.6 in HDP 2.5.3 through Java API, Sqoop.runTool, using HCATALOG

New Contributor

When we call Sqoop.runTool to import from an Oracle Database. once time the Map And Reduce is launcher it's unable to find the HCATALOG classes:

2017-09-18 09:04:47,546 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.mapreduce.HCatOutputFormat not found
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.mapreduce.HCatOutputFormat not found
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:519)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:499)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1598)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:499)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:285)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1556)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1553)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1486)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.mapreduce.HCatOutputFormat not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2214)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getOutputFormatClass(JobContextImpl.java:222)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:515)
	... 11 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.mapreduce.HCatOutputFormat not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2120)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2212)
	... 13 more


SOLUTION:


finally, I found the solution setting in the configuration object before call to Sqoop.runTool(s[], conf) the hcatalog jars in a hdfs path


			conf.set("tmpjars", "/lib/hive-exec-1.2.1.2.3.4.0-3485.jar,/lib/hive-hcatalog-core-1.2.1.2.3.4.0-3485.jar,/lib/hive-jdbc-1.2.1.2.3.4.0-3485.jar...");