Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive Metastore RPC fails on Sentry HDFS MetastorePlugin after upgrade to CDH 5.8.2 on Oracle BDA

Highlighted

Hive Metastore RPC fails on Sentry HDFS MetastorePlugin after upgrade to CDH 5.8.2 on Oracle BDA

Explorer

We have recently upgraded an Oracle BDA from CDH 5.8.0 to CDH 5.8.2, after the upgrade we encounter an error with Hive Metastore RPC when we run a DDL command.

 

Every time we run a DDL command the command fails with an error with a NullPointerPointer.

Anyway the command was be executed successfully, in Hive, Beeline, Impala-Shell and Hue we can work as the error was not raised.

Unfortunately some tools, like Sqoop, stop to work after the error.

 

Looking into HiveMetastore Log we find this entry:

2016-11-14 12:43:50,043 ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-6-thread-4]: MetaException(message:java.lang.NullPointerException)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newMetaException(HiveMetaStore.java:5600)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database(HiveMetaStore.java:967)
        at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:138)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
        at com.sun.proxy.$Proxy8.create_database(Unknown Source)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_database.getResult(ThriftHiveMetastore.java:8863)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_database.getResult(ThriftHiveMetastore.java:8847)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:735)
        at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:730)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
        at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:730)
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
        at org.apache.sentry.hdfs.MetastorePlugin.applyLocal(MetastorePlugin.java:344)
        at org.apache.sentry.hdfs.MetastorePlugin.processUpdate(MetastorePlugin.java:375)
        at org.apache.sentry.hdfs.MetastorePlugin.notifySentryAndApplyLocal(MetastorePlugin.java:352)
        at org.apache.sentry.hdfs.MetastorePlugin.addPath(MetastorePlugin.java:229)
        at org.apache.sentry.binding.metastore.SentryMetastorePostEventListener.onCreateDatabase(SentryMetastorePostEventListener.java:163)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database_core(HiveMetaStore.java:927)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database(HiveMetaStore.java:956)
        ... 20 more

Where we found that the Sentry Plugin raises the error. 

 

Il looks the same error as this sentry issue: [SENTRY-1270] Improve error handling - Database with malformed URI causes NPE in HMS plugin during D....

 

To be honest I do not know if is an Oracle BDA issue or a CDH issue.