Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Hive Metastore fails to start for a newly installed cluster.

SOLVED Go to solution

Hive Metastore fails to start for a newly installed cluster.

Explorer

I have a 5 node cluster, where I have deployed CDH-5.9.0-1.cdh5.9.0.p0.23-el7.parcel

Other details: Java Version: 1.8., CentOS Linux release 7.3

 

Now, all the services are up and running except for Hive- Hive Metastore, and which in turn results for Impala's bad health. The major issue is that the Hive metastore is not coming up and throws the below error:

java.lang.NullPointerException
at org.apache.hadoop.hive.metastore.ObjectStore.getObjectCount(ObjectStore.java:1103)
at org.apache.hadoop.hive.metastore.ObjectStore.getTableCount(ObjectStore.java:1081)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:103)
at com.sun.proxy.$Proxy7.getTableCount(Unknown Source)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.updateMetrics(HiveMetaStore.java:6244)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:528)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6280)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6275)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:6525)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:6452)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Exception in thread "main" java.lang.NullPointerException

 

On further investigating the logs, I found the message as below:

Can't open /opt/cm5.9.0/run/cloudera-scm-agent/process/{PID}-hive-HIVEMETASTORE/supervisor.conf: Permission denied.         But i think this "Permission denied" isn't the cause,because this "Permission denied" is appearing while starting HBase too,so, i think this "Permission denied" isn't the cause. By the way, hive MetaStore link to Mysql metaDataBase  is ok, it can create table and insert data automatically.

 

This trouble is the same as here   , Nobody reply except me.

 

Anyone faced the same issue and got this fixed or has any idea about the issue..THANKS!

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Hive Metastore fails to start for a newly installed cluster.

Explorer

hello everyone. i had solved the problem.

the cause is MySql database encoding, run the following command solved the problem "alter database hive character set latin1" . dropping the "hive" database and regenerate "hive" database. Thanks !.

3 REPLIES 3

Re: Hive Metastore fails to start for a newly installed cluster.

Master Collaborator
I dont know the solution, but I think cloudera manager agent which is running under root is starting these processes and sets the correct permissions. Now I could imagine, that maybe the cloudera-scm-agent is not running under root, or maybe the permissions are set wrongly.

I would check the exact process directory of the hive metastore.
I would also verify in processes, that just one copy of HiveMetastore is running (to eliminate the possibility that two processes are running in the same time)
Try to restart the cloudera-scm-agent.

Re: Hive Metastore fails to start for a newly installed cluster.

Champion

check if there are multiple supervisor runining on that host using the below command . 

 

ps aux |grep supervisor

 

if so  you will  need to kill it and then start the cloudera -scm  agent and kick off the hive metastore . 

Highlighted

Re: Hive Metastore fails to start for a newly installed cluster.

Explorer

hello everyone. i had solved the problem.

the cause is MySql database encoding, run the following command solved the problem "alter database hive character set latin1" . dropping the "hive" database and regenerate "hive" database. Thanks !.