Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hive services fails to start especially Hive metastore

avatar
Explorer

Hi 

 

This is in continuation with my earlier question (https://community.cloudera.com/t5/Support-Questions/Services-fail-to-start-gets-stuck-on-HDP-3-0-san...) about services failing to start after certain time. However following the suggestions I have stopped all the services and just trying to run zookeeper and Hive and still the Hive metastore service fails to start.

 

5.PNG6.PNG

 

 

Any solution to this issue and what might be causing this?

1 ACCEPTED SOLUTION

avatar
Master Mentor

@tyadav 
Every service provides a "metainfo.xml" file which is used by ambari to determine the <dependencies> between individual service/components.

For example in case of HIVE service you can see a "metainfo.xml" file in the following location.

/var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml
/var/lib/ambari-server/resources/stacks/HDP/3.0/services/HIVE/metainfo.xml

 

You can find the <dependencies> tab inside this file which talks about the component dependencies. For example you can see the "HIVE_SERVER" dependencies as following:

        <component>
          <name>HIVE_SERVER</name>
          <displayName>HiveServer2</displayName>
          <category>MASTER</category>
          <cardinality>1</cardinality>
          <versionAdvertised>true</versionAdvertised>
          <reassignAllowed>true</reassignAllowed>
          <clientsToUpdateConfigs></clientsToUpdateConfigs>
          <dependencies>
            <dependency>
              <name>ZOOKEEPER/ZOOKEEPER_SERVER</name>
              <scope>cluster</scope>
              <auto-deploy>
                <enabled>true</enabled>
                <co-locate>HIVE/HIVE_SERVER</co-locate>
              </auto-deploy>
            </dependency>
            <dependency>
              <name>YARN/YARN_CLIENT</name>
              <scope>host</scope>
              <auto-deploy>
                <enabled>true</enabled>
              </auto-deploy>
            </dependency>
            <dependency>
              <name>MAPREDUCE2/MAPREDUCE2_CLIENT</name>
              <scope>host</scope>
              <auto-deploy>
                <enabled>true</enabled>
              </auto-deploy>
            </dependency>
          </dependencies>

.

Similarly you can also find the dependencies using the following kind of Ambari API call:

# curl -sk -u admin:admin -H "X-Requested-By: ambari" -X GET http://$AMBARI_HOST:8080/api/v1/stacks/HDP/versions/3.1/services/HIVE   | grep -A5 'required_services'

    "required_services" : [
      "ZOOKEEPER",
      "HDFS",
      "YARN",
      "TEZ"
    ],

.

.

If your question is answered then, Please make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

 

View solution in original post

4 REPLIES 4

avatar
Master Mentor

@tyadav 

Some of the hive tables that are created in HDFS  might not be accessible if the HDFS is down. Hive Service has a dependency to HDFS/YARN/MapReduce.   So are you able to start the HDFS/YARN/MapReduce service first?

 

Also If HiveMetastore service is not coming up then can you please share the HiveMetastore log , including the Operational Log from Ambari UI where we can see the failure cause ?

 

avatar
Explorer

Hi,

 

Yes HDFS and Zookeper were running while writing the original post (not sure about Mapreduce and YARN though). However after shutting down the VM and restarting everything including Hive works as of now.

 

Is there a source/matrix somewhere to read about the inter-dependencies of these services because it is important to know which services are dependent on each other and to what extent especially for a beginner and also while talking about saving the memory space while running limited services?

 

avatar
Master Mentor

@tyadav 
Every service provides a "metainfo.xml" file which is used by ambari to determine the <dependencies> between individual service/components.

For example in case of HIVE service you can see a "metainfo.xml" file in the following location.

/var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml
/var/lib/ambari-server/resources/stacks/HDP/3.0/services/HIVE/metainfo.xml

 

You can find the <dependencies> tab inside this file which talks about the component dependencies. For example you can see the "HIVE_SERVER" dependencies as following:

        <component>
          <name>HIVE_SERVER</name>
          <displayName>HiveServer2</displayName>
          <category>MASTER</category>
          <cardinality>1</cardinality>
          <versionAdvertised>true</versionAdvertised>
          <reassignAllowed>true</reassignAllowed>
          <clientsToUpdateConfigs></clientsToUpdateConfigs>
          <dependencies>
            <dependency>
              <name>ZOOKEEPER/ZOOKEEPER_SERVER</name>
              <scope>cluster</scope>
              <auto-deploy>
                <enabled>true</enabled>
                <co-locate>HIVE/HIVE_SERVER</co-locate>
              </auto-deploy>
            </dependency>
            <dependency>
              <name>YARN/YARN_CLIENT</name>
              <scope>host</scope>
              <auto-deploy>
                <enabled>true</enabled>
              </auto-deploy>
            </dependency>
            <dependency>
              <name>MAPREDUCE2/MAPREDUCE2_CLIENT</name>
              <scope>host</scope>
              <auto-deploy>
                <enabled>true</enabled>
              </auto-deploy>
            </dependency>
          </dependencies>

.

Similarly you can also find the dependencies using the following kind of Ambari API call:

# curl -sk -u admin:admin -H "X-Requested-By: ambari" -X GET http://$AMBARI_HOST:8080/api/v1/stacks/HDP/versions/3.1/services/HIVE   | grep -A5 'required_services'

    "required_services" : [
      "ZOOKEEPER",
      "HDFS",
      "YARN",
      "TEZ"
    ],

.

.

If your question is answered then, Please make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

 

avatar
Explorer

@jsensharma  thanks. That helps.