Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive not resolving namenode HA

Solved Go to solution
Highlighted

Hive not resolving namenode HA

Expert Contributor

Hello,

After enabling HA on the name node, hive is unable to access the hive databases whose metastore name got updated to the new FS path:

Error: Error while compiling statement: FAILED: SemanticException java.lang.IllegalArgumentException: java.net.UnknownHostException: cluster1 (state=42000,code=40000)

hive metastore tool shows:

hive --service metatool -listFSRootListing FS Roots..

hdfs://cluster1/apps/hive/warehouse/test2.db

hdfs://cluster1/apps/hive/warehouse/raw.db

hdfs://cluster1/apps/hive/warehouse/test.db

hdfs://cluster1/apps/hive/warehouse

hdfs://cluster1/apps/hive/warehouse/lookup.db

cluster1 is the correct fs.defaultFS that was setup during HA.

<property>

<name>fs.defaultFS</name>

<value>hdfs://cluster1</value>

<final>true</final>

</property>

If I create a new database in hive, it gets created using the actual name node host name:

0: jdbc:hive2://ip-10-555-2-555.ec2.internal:> create database test3;

No rows affected (0.148 seconds)

Listing FS Roots..

hdfs://ip-10-123-5-42.ec2.internal:8020/apps/hive/warehouse/test3.db

any ideas on what could be missing? hdfs client works fine with the HA name.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Hive not resolving namenode HA

@mliem

Please refer to below link for configuring namenode HA. Make sure that step#15 is executed

https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.1/bk_Ambari_Users_Guide/content/_how_to_confi...

Below is the details of Step#15

If you are using Hive, you must manually change the Hive Metastore FS root to point to the Nameservice URI instead of the NameNode URI. You created the Nameservice ID in the Get Started step.
Check the current FS root. On the Hive host:
hive --config /etc/hive/conf.server --service metatool -listFSRoot
The output looks similar to the following: Listing FS Roots... hdfs://<namenode-host>/apps/hive/warehouse
Use this command to change the FS root:
$ hive --config /etc/hive/conf.server --service metatool -updateLocation <new-location><old-location>
For example, where the Nameservice ID is mycluster:
$ hive --config /etc/hive/conf.server --service metatool -updateLocation hdfs://mycluster/apps/hive/warehouse hdfs://c6401.ambari.apache.org/apps/hive/warehouse
The output looks similar to the following:
Successfully updated the following locations...Updated X records in SDS table

View solution in original post

2 REPLIES 2

Re: Hive not resolving namenode HA

@mliem

Please refer to below link for configuring namenode HA. Make sure that step#15 is executed

https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.1/bk_Ambari_Users_Guide/content/_how_to_confi...

Below is the details of Step#15

If you are using Hive, you must manually change the Hive Metastore FS root to point to the Nameservice URI instead of the NameNode URI. You created the Nameservice ID in the Get Started step.
Check the current FS root. On the Hive host:
hive --config /etc/hive/conf.server --service metatool -listFSRoot
The output looks similar to the following: Listing FS Roots... hdfs://<namenode-host>/apps/hive/warehouse
Use this command to change the FS root:
$ hive --config /etc/hive/conf.server --service metatool -updateLocation <new-location><old-location>
For example, where the Nameservice ID is mycluster:
$ hive --config /etc/hive/conf.server --service metatool -updateLocation hdfs://mycluster/apps/hive/warehouse hdfs://c6401.ambari.apache.org/apps/hive/warehouse
The output looks similar to the following:
Successfully updated the following locations...Updated X records in SDS table

View solution in original post

Highlighted

Re: Hive not resolving namenode HA

Expert Contributor
@santoshsb

They are already configured to point to the nameservice URI. See my first screenshot:

hive --service metatool -listFSRootListing FS Roots.

hdfs://cluster1/apps/hive/warehouse/test2.d

hdfs://cluster1/apps/hive/warehouse/raw.db

hdfs://cluster1/apps/hive/warehouse/test.db

hdfs://cluster1/apps/hive/warehouse

hdfs://cluster1/apps/hive/warehouse/lookup.db

Don't have an account?
Coming from Hortonworks? Activate your account here