Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

avatar
Expert Contributor

hd-hive.pngI am new to HDinsight hadoop cluster , when i have open and Ambari ->hive view>query and execute any hive query(show tables etc..) i am getting error: java.net:UnknowhostException: namenode

Note: I am able to trigger hive query in HIVE cli .

HDP 2.7

hive 1.2

1 ACCEPTED SOLUTION

avatar

@rama

Hive view will internally use: webhdfs file system uri So please check if the following property is set properly or not?

Click HDFS > Configs > Advanced > Advanced hdfs-site > dfs.namenode.http-address. When you enter the value in the view definition, pre-pend "webhdfs://" to the value you find in the advanced HDFS configuration settings.

For example, webhdfs://c6401.ambari.apache.org:50070

Please see: https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/section_get_v...

View solution in original post

15 REPLIES 15

avatar
Master Mentor

in your Hive view, make sure webhdfs.url property is set to correct namenode url, I.e. webhdfs://fqdn:8020 or in case your cluster is HA, webhdfs://nameservices where nameservices is the dfs.nameservices property.

avatar
Expert Contributor

Thanks you so much@Artem Ervits

Can you help me find webhdfs.url property ..where i can find these property?

avatar
Master Mentor

Look in hdfs-site.xml in advanced Ambari config for hdfs service

avatar
Expert Contributor

@Artem Ervits

sorry @Artem Ervits

Still i am unable to find the webhdfs.url property in hdfs_site.xml in hdfs service

I have attached hdfs services screen short please seehdfs-site1.pnghdfs-site2.pnghdfs-site3.pnghdfs-site4.png

avatar
Expert Contributor

@Artem Ervits

sorry @Artem Ervits

Still i am unable to find the webhdfs.url property in hdfs_site.xml in hdfs service

I have attached hdfs services screen short please seehdfs-site1.pnghdfs-site2.pnghdfs-site3.pnghdfs-site4.png

avatar
Master Mentor

I should've realized that you're on Azure and using WASB, I am not 100 % sure but see if you can take the external address of the FS.defaultFS property and add to Hive view. You need to fill out the following properties in your Hive view with their associated values. This is in case it is HA, in case you also have kerberos, you need to add those properties as well. Keep in mind you need public address for each URL not the ones that says internal in their name. I have example REST API calls to create Ambari views in this script, properties in the curl calls are needed to setup views to function https://github.com/dbist/ambari-chef in that link go to recipes then ambari_views_setup.rb file for examples

"webhdfs.client.failover.proxy.provider" : "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider",      "webhdfs.ha.namenode.http-address.nn1" : "u1201.ambari.apache.org:50070",      "webhdfs.ha.namenode.http-address.nn2" : "u1201.ambari.apache.org:50070",      "webhdfs.ha.namenode.https-address.nn1" : null,      "webhdfs.ha.namenode.https-address.nn2" : null,      "webhdfs.ha.namenode.rpc-address.nn1" : "u1201.ambari.apache.org:8020",      "webhdfs.ha.namenode.rpc-address.nn2" : "u1202.ambari.apache.org:8020",      "webhdfs.ha.namenodes.list" : "nn1,nn2",      "webhdfs.nameservices" : "hacluster",      "webhdfs.url" : "webhdfs://hacluster",      "hive.host" : "u1203.ambari.apache.org",      "hive.http.path" : "cliservice",      "hive.http.port" : "10001",      "hive.metastore.warehouse.dir" : "/apps/hive/warehouse",      "hive.port" : "10000",      "hive.transport.mode" : "binary",      "yarn.ats.url" : "http://u1202.ambari.apache.org:8188",      "yarn.resourcemanager.url" : "u1202.ambari.apache.org:8088"

avatar
Expert Contributor

Thanks you so much @Artem Ervits

I will check and post result..

avatar
Master Mentor

Also make sure to open endpoints to outside for the ports you're using in azure portal.

avatar

@rama

Hive view will internally use: webhdfs file system uri So please check if the following property is set properly or not?

Click HDFS > Configs > Advanced > Advanced hdfs-site > dfs.namenode.http-address. When you enter the value in the view definition, pre-pend "webhdfs://" to the value you find in the advanced HDFS configuration settings.

For example, webhdfs://c6401.ambari.apache.org:50070

Please see: https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/section_get_v...