Hive Metastore fails to start for a newly installed cluster.

eniac27427 — Fri, 16 Sep 2022 12:36:16 GMT

I have a 5 node cluster, where I have deployed CDH-5.9.0-1.cdh5.9.0.p0.23-el7.parcel

Other details: Java Version: 1.8., CentOS Linux release 7.3

Now, all the services are up and running except for Hive- Hive Metastore, and which in turn results for Impala's bad health. The major issue is that the Hive metastore is not coming up and throws the below error:

java.lang.NullPointerException
at org.apache.hadoop.hive.metastore.ObjectStore.getObjectCount(ObjectStore.java:1103)
at org.apache.hadoop.hive.metastore.ObjectStore.getTableCount(ObjectStore.java:1081)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:103)
at com.sun.proxy.$Proxy7.getTableCount(Unknown Source)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.updateMetrics(HiveMetaStore.java:6244)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:528)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6280)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6275)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:6525)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:6452)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Exception in thread "main" java.lang.NullPointerException

On further investigating the logs, I found the message as below:

Can't open /opt/cm5.9.0/run/cloudera-scm-agent/process/{PID}-hive-HIVEMETASTORE/supervisor.conf: Permission denied. But i think this "Permission denied" isn't the cause,because this "Permission denied" is appearing while starting HBase too,so, i think this "Permission denied" isn't the cause. By the way, hive MetaStore link to Mysql metaDataBase is ok, it can create table and insert data automatically.

This trouble is the same as here , Nobody reply except me.

Anyone faced the same issue and got this fixed or has any idea about the issue..THANKS!

Re: Hive Metastore fails to start for a newly installed cluster.

Tomas79 — Wed, 06 Dec 2017 14:07:25 GMT

I dont know the solution, but I think cloudera manager agent which is running under root is starting these processes and sets the correct permissions. Now I could imagine, that maybe the cloudera-scm-agent is not running under root, or maybe the permissions are set wrongly.

I would check the exact process directory of the hive metastore.
I would also verify in processes, that just one copy of HiveMetastore is running (to eliminate the possibility that two processes are running in the same time)
Try to restart the cloudera-scm-agent.

Re: Hive Metastore fails to start for a newly installed cluster.

csguna — Thu, 07 Dec 2017 15:35:18 GMT

check if there are multiple supervisor runining on that host using the below command .

ps aux |grep supervisor

if so you will need to kill it and then start the cloudera -scm agent and kick off the hive metastore .

Re: Hive Metastore fails to start for a newly installed cluster.

eniac27427 — Sat, 09 Dec 2017 14:51:31 GMT

hello everyone. i had solved the problem.

the cause is MySql database encoding, run the following command solved the problem "alter database hive character set latin1" . dropping the "hive" database and regenerate "hive" database. Thanks !.

question Re: Hive Metastore fails to start for a newly installed cluster. in Archives of Support Questions (Read Only)

Hive Metastore fails to start for a newly installed cluster.

Re: Hive Metastore fails to start for a newly installed cluster.

Re: Hive Metastore fails to start for a newly installed cluster.

Re: Hive Metastore fails to start for a newly installed cluster.