Error while compiling statement: FAILED: Hive Internal Error:

[#1]

Hello,

I'm facing issue while executing query (show databases;) on hive.

Following is my setup:

Cluster : CDP-7.1.7

Components Installed : HDFS, Yarn, Yarn Queue Manager, Oozie, Schema Registry, Hive, Ziplin, Spark, Tez and Zookeeper. 

I have accessed hive using beeline, but when I tried to execute 'show databases' it is giving following error :

ERROR : FAILED: Hive Internal Error: org.apache.hadoop.hive.ql.metadata.HiveException(Error while invoking PostHook. hooks: java.lang.RuntimeException: java.lang.NullPointerException
at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:50)
at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.getPipeline(AbstractHiveExecHookContext.java:86)
at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.run(AbstractHiveExecHookContext.java:69)
at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296)
at org.apache.hadoop.hive.ql.HookRunner.runPostExecHooks(HookRunner.java:278)
at org.apache.hadoop.hive.ql.Executor.postExecutionActions(Executor.java:477)
at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:111)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:749)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:504)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:498)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166)
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226)
at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:88)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:327)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:345)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at java.io.File.<init>(File.java:277)
at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:43)
... 24 more
)
org.apache.hadoop.hive.ql.metadata.HiveException: Error while invoking PostHook. hooks: java.lang.RuntimeException: java.lang.NullPointerException
at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:50)
at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.getPipeline(AbstractHiveExecHookContext.java:86)
at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.run(AbstractHiveExecHookContext.java:69)
at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296)
at org.apache.hadoop.hive.ql.HookRunner.runPostExecHooks(HookRunner.java:278)
at org.apache.hadoop.hive.ql.Executor.postExecutionActions(Executor.java:477)
at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:111)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:749)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:504)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:498)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166)
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226)
at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:88)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:327)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:345)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at java.io.File.<init>(File.java:277)
at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:43)
... 24 more

at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:302)
at org.apache.hadoop.hive.ql.HookRunner.runPostExecHooks(HookRunner.java:278)
at org.apache.hadoop.hive.ql.Executor.postExecutionActions(Executor.java:477)
at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:111)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:749)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:504)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:498)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166)
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226)
at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:88)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:327)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:345)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:50)
at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.getPipeline(AbstractHiveExecHookContext.java:86)
at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.run(AbstractHiveExecHookContext.java:69)
at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296)
... 21 more
Caused by: java.lang.NullPointerException
at java.io.File.<init>(File.java:277)
at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:43)
... 24 more

INFO : Completed executing command(queryId=hive_20220330193703_a3c2039f-3446-4d1c-8ed6-b4e88ac77771); Time taken: 0.051 seconds
INFO : OK
Error: Error while compiling statement: FAILED: Hive Internal Error: org.apache.hadoop.hive.ql.metadata.HiveException(Error while invoking PostHook. hooks: java.lang.RuntimeException: java.lang.NullPointerException
at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:50)
at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.getPipeline(AbstractHiveExecHookContext.java:86)
at com.cloudera.navigator.audit.hive.AbstractHiveExecHookContext.run(AbstractHiveExecHookContext.java:69)
at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296)
at org.apache.hadoop.hive.ql.HookRunner.runPostExecHooks(HookRunner.java:278)
at org.apache.hadoop.hive.ql.Executor.postExecutionActions(Executor.java:477)
at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:111)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:749)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:504)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:498)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166)
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226)
at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:88)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:327)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:345)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at java.io.File.<init>(File.java:277)
at com.cloudera.navigator.audit.hive.HiveAuditPipeline.getInstance(HiveAuditPipeline.java:43)
... 24 more
) (state=08S01,code=12)

Can anyone please help me this?

Thanks. 

[#2]

Hi @dmharshit ,

This error can occures if you have deployed the "Hive Server 2" and "WebHCat" roles under the Hive Service.

Can you stop and delete the "Hive Server 2" and "WebHCat" instances from Cloudera Manager -> Hive -> Instances? 

Restart if any stale configurations are present in the cluster and try the hive query.

Regards,

Chethan YM

[#3]

After following the steps above, the problem still exists.

[#4]

Could you please share screenshots of the Instance tab from both Hive and Hive on Tez services? Based on our experience, the following action should resolve the issue.

1. Stop and delete the "HiveServer2" and "WebHCat" instances from Cloudera Manager > Hive > Instances
2. Check that the role "HiveServer2" is present in Cloudera Manager > Hive on Tez > Instances
3. Verify the Configuration page for 'Hive on Tez'; ensure that 'Hive Metastore Connector' is pointed at the Hive service that is running the Hive Metastore
4. Restart the stale services