Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive action failing in Oozie workflow

Hive action failing in Oozie workflow

Explorer

We are facing issue with running oozie jobs which involves hive action. We are running Couldera 5.1. below is the error.

 

java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.HiveMain not found

 

ailing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exception invoking main(), java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.HiveMain not found
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.HiveMain not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1897)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:224)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.HiveMain not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1803)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	... 9 more
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.impl.MetricsSystemImpl).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

 

Please find the below details.

 

We do have required jar files present in sharedlib.

-rwxrwxrwx 3 oozie oozie 36896 2015-01-06 12:28 /user/oozie/share/lib/lib_20150106122803/hive/hive-cli-0.12.0-cdh5.1.0.jar
-rwxrwxrwx 3 oozie oozie 110674 2015-01-06 12:28 /user/oozie/share/lib/lib_20150106122803/hive/hive-common-0.12.0-cdh5.1.0.jar
-rwxrwxrwx 3 oozie oozie 115527 2015-01-06 12:28 /user/oozie/share/lib/lib_20150106122803/hive/hive-contrib-0.12.0-cdh5.1.0.jar
-rwxrwxrwx 3 oozie oozie 10153799 2015-01-06 12:28 /user/oozie/share/lib/lib_20150106122803/hive/hive-exec-0.12.0-cdh5.1.0.jar
-rwxrwxrwx 3 oozie oozie 453776 2015-02-04 12:35 /user/oozie/share/lib/lib_20150106122803/hive/hive-hcatalog-core-0.12.0-cdh5.1.0.jar
-rwxrwxrwx 3 oozie oozie 66320 2015-02-04 12:36 /user/oozie/share/lib/lib_20150106122803/hive/hive-hcatalog-server-extensions-0.12.0-cdh5.1.0.jar
-rwxrwxrwx 3 oozie oozie 3451957 2015-01-06 12:28 /user/oozie/share/lib/lib_20150106122803/hive/hive-metastore-0.12.0-cdh5.1.0.jar
-rwxrwxrwx 3 oozie oozie 731658 2015-01-06 12:28 /user/oozie/share/lib/lib_20150106122803/hive/hive-serde-0.12.0-cdh5.1.0.jar
-rwxrwxrwx 3 oozie oozie 1758419 2015-01-06 12:28 /user/oozie/share/lib/lib_20150106122803/hive/hive-service-0.12.0-cdh5.1.0.jar
-rwxrwxrwx 3 oozie oozie 7568 2015-01-06 12:28 /user/oozie/share/lib/lib_20150106122803/hive/hive-shims-0.12.0-cdh5.1.0.jar
-rwxrwxrwx 3 oozie oozie 31617 2015-01-06 12:28 /user/oozie/share/lib/lib_20150106122803/hive/hive-shims-0.23-0.12.0-cdh5.1.0.jar
-rwxrwxrwx 3 oozie oozie 36765 2015-01-06 12:28 /user/oozie/share/lib/lib_20150106122803/hive/hive-shims-common-0.12.0-cdh5.1.0.jar
-rwxrwxrwx 3 oozie oozie 75455 2015-01-06 12:28 /user/oozie/share/lib/lib_20150106122803/hive/hive-shims-common-secure-0.12.0-cdh5.1.0.jar


cat job.properties
#hadoop configurations
namenode=hdfs://cdh-node1:8020
jobTracker=cdh-node1
# Path in hdfs where the coordinator.xml is located. For ecds the co-ordinator xml is ecds_fileMonitorCoordinator.xml
#oozie.coord.application.path=/ecds/pat/data/oozie/workflow/coordinator.xml
#following properties are used by coordinator.xml
#frequence of execution. Frequency is expressed in minutes.
#frequency=60
#startTime=2014-08-06T22:40Z
#endTime=2015-08-07T21:15Z
#timezone=UTC
#workflow that need to be cordinated from co-ordinator xml. For ecds, co-ordinator.xml invokes ecds file monitor workflow
oozie.wf.application.path=/workspace/ecrr/pat/conf/oozie/workflow/ecrr_hive_workflow.xml
#libpath where oozie can locate the resources
#ozie.libpath=
keytabFilename=adminuser.keytab
jassFilename=ab-perf-jaas.conf
oozie.wf.validate.ForkJoin=false

 

 

5 REPLIES 5

Re: Hive action failing in Oozie workflow

Expert Contributor

Hey,

 

You need to include "oozie.use.system.libpath=true" in your job.properties or the workflow won't pick up the Oozie share libs.  Can you give this a try?

 

Thanks

Chris

Re: Hive action failing in Oozie workflow

Explorer

We have mentioned that in workflow.xml

 

<property>
<name>oozie.use.system.libpath</name>
<value>true</value>
</property>

<property>
<name>oozie.libpath</name>
<value>/user/oozie/share/lib/lib_20150106122803/</value>
</property>

Re: Hive action failing in Oozie workflow

Explorer

We tried runing after adding that in job.properties. But still HiveMain not found error

Re: Hive action failing in Oozie workflow

New Contributor

I am also facing the same issue ...did any one knowhow to resolvethis issue...if run shell script througing this Java Main not found error ..when I run the workflow throughing as sharlib not found error.. anyhelp is appriciated..

Highlighted

Re: Hive action failing in Oozie workflow

New Contributor

I faced similar issue and adding the below lines to job.properties file solved my problem.

 

oozie.use.system.libpath=true
oozie.libpath=${nameNode}/user/oozie/share/lib

 

Note: nameNode = http://<hostname_where_namenode_is_running>:8020 is already been set in job.properties file.

 

It might also work if above properties are added under configuration tag in workflow.xml