question Hive not resolving namenode HA in Archives of Support Questions (Read Only)

Hive not resolving namenode HA

mliem — Wed, 12 Oct 2016 05:36:39 GMT

Hello,

After enabling HA on the name node, hive is unable to access the hive databases whose metastore name got updated to the new FS path:

Error: Error while compiling statement: FAILED: SemanticException java.lang.IllegalArgumentException: java.net.UnknownHostException: cluster1 (state=42000,code=40000)

hive metastore tool shows:

hive --service metatool -listFSRootListing FS Roots..

hdfs://cluster1/apps/hive/warehouse/test2.db

hdfs://cluster1/apps/hive/warehouse/raw.db

hdfs://cluster1/apps/hive/warehouse/test.db

hdfs://cluster1/apps/hive/warehouse

hdfs://cluster1/apps/hive/warehouse/lookup.db

cluster1 is the correct fs.defaultFS that was setup during HA.

<property>

<name>fs.defaultFS</name>

<value>hdfs://cluster1</value>

<final>true</final>

</property>

If I create a new database in hive, it gets created using the actual name node host name:

0: jdbc:hive2://ip-10-555-2-555.ec2.internal:> create database test3;

No rows affected (0.148 seconds)

Listing FS Roots..

hdfs://ip-10-123-5-42.ec2.internal:8020/apps/hive/warehouse/test3.db

any ideas on what could be missing? hdfs client works fine with the HA name.

Re: Hive not resolving namenode HA

sgowda — Wed, 12 Oct 2016 13:48:27 GMT

@mliem

Please refer to below link for configuring namenode HA. Make sure that step#15 is executed

https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.1/bk_Ambari_Users_Guide/content/_how_to_configure_namenode_high_availability.html

Below is the details of Step#15

If you are using Hive, you must manually change the Hive Metastore FS root to point to the Nameservice URI instead of the NameNode URI. You created the Nameservice ID in the Get Started step.
Check the current FS root. On the Hive host:
hive --config /etc/hive/conf.server --service metatool -listFSRoot
The output looks similar to the following: Listing FS Roots... hdfs://<namenode-host>/apps/hive/warehouse
Use this command to change the FS root:
$ hive --config /etc/hive/conf.server --service metatool -updateLocation <new-location><old-location>
For example, where the Nameservice ID is mycluster:
$ hive --config /etc/hive/conf.server --service metatool -updateLocation hdfs://mycluster/apps/hive/warehouse hdfs://c6401.ambari.apache.org/apps/hive/warehouse
The output looks similar to the following:
Successfully updated the following locations...Updated X records in SDS table

Re: Hive not resolving namenode HA

mliem — Wed, 12 Oct 2016 21:43:32 GMT

@santoshsb

They are already configured to point to the nameservice URI. See my first screenshot:

hive --service metatool -listFSRootListing FS Roots.

hdfs://cluster1/apps/hive/warehouse/test2.d

hdfs://cluster1/apps/hive/warehouse/raw.db

hdfs://cluster1/apps/hive/warehouse/test.db

hdfs://cluster1/apps/hive/warehouse

hdfs://cluster1/apps/hive/warehouse/lookup.db