Hive Metastore fails to start for a newly installed cluster.

[#1]

I have a 5 node cluster, where I have deployed CDH-5.9.0-1.cdh5.9.0.p0.23-el7.parcel

Other details: Java Version: 1.8., CentOS Linux release 7.3

Now, all the services are up and running except for Hive- Hive Metastore, and which in turn results for Impala's bad health. The major issue is that the Hive metastore is not coming up and throws the below error:

java.lang.NullPointerException
at org.apache.hadoop.hive.metastore.ObjectStore.getObjectCount(ObjectStore.java:1103)
at org.apache.hadoop.hive.metastore.ObjectStore.getTableCount(ObjectStore.java:1081)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:103)
at com.sun.proxy.$Proxy7.getTableCount(Unknown Source)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.updateMetrics(HiveMetaStore.java:6244)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:528)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6280)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6275)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:6525)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:6452)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Exception in thread "main" java.lang.NullPointerException

On further investigating the logs, I found the message as below:

Can't open /opt/cm5.9.0/run/cloudera-scm-agent/process/{PID}-hive-HIVEMETASTORE/supervisor.conf: Permission denied.         But i think this "Permission denied" isn't the cause,because this "Permission denied" is appearing while starting HBase too,so, i think this "Permission denied" isn't the cause. By the way, hive MetaStore link to Mysql metaDataBase  is ok, it can create table and insert data automatically.

This trouble is the same as here   , Nobody reply except me.

Anyone faced the same issue and got this fixed or has any idea about the issue..THANKS!

[#2]

I dont know the solution, but I think cloudera manager agent which is running under root is starting these processes and sets the correct permissions. Now I could imagine, that maybe the cloudera-scm-agent is not running under root, or maybe the permissions are set wrongly.

I would check the exact process directory of the hive metastore.
I would also verify in processes, that just one copy of HiveMetastore is running (to eliminate the possibility that two processes are running in the same time)
Try to restart the cloudera-scm-agent.

[#3]

check if there are multiple supervisor runining on that host using the below command . 

ps aux |grep supervisor

if so  you will  need to kill it and then start the cloudera -scm  agent and kick off the hive metastore . 

[#4 · Accepted solution]

hello everyone. i had solved the problem.

the cause is MySql database encoding, run the following command solved the problem "alter database hive character set latin1" . dropping the "hive" database and regenerate "hive" database. Thanks !.