Hive Metastore - Method not found

[#1]

HDP 2.3

------------------

I am getting this error intermittently. Sometime the Hive action in oozie works fine.

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], main() threw exception, org.apache.hadoop.hive.metastore.IMetaStoreClient.isLocalMetaStore()Z
java.lang.NoSuchMethodError: org.apache.hadoop.hive.metastore.IMetaStoreClient.isLocalMetaStore()Z
	at org.apache.hadoop.hive.ql.session.SessionState.unCacheDataNucleusClassLoaders(SessionState.java:1474)
	at org.apache.hadoop.hive.ql.session.SessionState.close(SessionState.java:1468)
	at org.apache.hadoop.hive.cli.CliSessionState.close(CliSessionState.java:66)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:683)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
	at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:306)
	at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:290)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
	at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:68)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:241)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Oozie Launcher failed, finishing Hadoop job gracefully

[#2]

@Sushil Saxena - Are you having duplicates jars (different version) in oozie sharelib for hive? can you please check that.

hadoop fs -ls /user/oozie/share/lib/lib_<date>/hive/

[#3]

I have the following :

hive-shims-0.20S-1.2.1.2.3.4.0-3485.jar

hive-shims-0.23-1.2.1.2.3.4.0-3485.jar

hive-shims-1.2.1.2.3.4.0-3485.jar

[#4]

@Sushil Saxena

It seems like there are likely to be multiple versions of hive jars. When I state multiple version of hive jars meaning there different hdp release hive jars in the oozie share lib location.

Was this a upgraded cluster from 2.1.x/2.2.x to 2.3.4 release?.

One thing that I can suggest here is to perform the following provided you have not copied over thirdparty jars like oracle jar/ mysql jar file to the oozie share lib location. If you have then you would need to copy over all the 3rd party jars again to hdfs.

From the oozie server host, using oozie user:

1. hdfs dfs -rm -r /user/oozie/share/lib -> to remove the lib folder completely

2. cd to /usr/hdp/current/oozie-client/bin

3. run "./oozie-setup.sh sharelib create -fs <get the fs.defaultFS from core-site.xml>"

4. Restart Oozie service

This should help to address the issue.