Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

Solved Go to solution

java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

Rising Star

hd-hive.pngI am new to HDinsight hadoop cluster , when i have open and Ambari ->hive view>query and execute any hive query(show tables etc..) i am getting error: java.net:UnknowhostException: namenode

Note: I am able to trigger hive query in HIVE cli .

HDP 2.7

hive 1.2

1 ACCEPTED SOLUTION

Accepted Solutions

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

@rama

Hive view will internally use: webhdfs file system uri So please check if the following property is set properly or not?

Click HDFS > Configs > Advanced > Advanced hdfs-site > dfs.namenode.http-address. When you enter the value in the view definition, pre-pend "webhdfs://" to the value you find in the advanced HDFS configuration settings.

For example, webhdfs://c6401.ambari.apache.org:50070

Please see: https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/section_get_v...

15 REPLIES 15

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

Mentor

in your Hive view, make sure webhdfs.url property is set to correct namenode url, I.e. webhdfs://fqdn:8020 or in case your cluster is HA, webhdfs://nameservices where nameservices is the dfs.nameservices property.

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

Rising Star

Thanks you so much@Artem Ervits

Can you help me find webhdfs.url property ..where i can find these property?

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

Mentor

Look in hdfs-site.xml in advanced Ambari config for hdfs service

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

Rising Star

@Artem Ervits

sorry @Artem Ervits

Still i am unable to find the webhdfs.url property in hdfs_site.xml in hdfs service

I have attached hdfs services screen short please seehdfs-site1.pnghdfs-site2.pnghdfs-site3.pnghdfs-site4.png

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

Rising Star

@Artem Ervits

sorry @Artem Ervits

Still i am unable to find the webhdfs.url property in hdfs_site.xml in hdfs service

I have attached hdfs services screen short please seehdfs-site1.pnghdfs-site2.pnghdfs-site3.pnghdfs-site4.png

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

Mentor

I should've realized that you're on Azure and using WASB, I am not 100 % sure but see if you can take the external address of the FS.defaultFS property and add to Hive view. You need to fill out the following properties in your Hive view with their associated values. This is in case it is HA, in case you also have kerberos, you need to add those properties as well. Keep in mind you need public address for each URL not the ones that says internal in their name. I have example REST API calls to create Ambari views in this script, properties in the curl calls are needed to setup views to function https://github.com/dbist/ambari-chef in that link go to recipes then ambari_views_setup.rb file for examples

"webhdfs.client.failover.proxy.provider" : "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider",      "webhdfs.ha.namenode.http-address.nn1" : "u1201.ambari.apache.org:50070",      "webhdfs.ha.namenode.http-address.nn2" : "u1201.ambari.apache.org:50070",      "webhdfs.ha.namenode.https-address.nn1" : null,      "webhdfs.ha.namenode.https-address.nn2" : null,      "webhdfs.ha.namenode.rpc-address.nn1" : "u1201.ambari.apache.org:8020",      "webhdfs.ha.namenode.rpc-address.nn2" : "u1202.ambari.apache.org:8020",      "webhdfs.ha.namenodes.list" : "nn1,nn2",      "webhdfs.nameservices" : "hacluster",      "webhdfs.url" : "webhdfs://hacluster",      "hive.host" : "u1203.ambari.apache.org",      "hive.http.path" : "cliservice",      "hive.http.port" : "10001",      "hive.metastore.warehouse.dir" : "/apps/hive/warehouse",      "hive.port" : "10000",      "hive.transport.mode" : "binary",      "yarn.ats.url" : "http://u1202.ambari.apache.org:8188",      "yarn.resourcemanager.url" : "u1202.ambari.apache.org:8088"

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

Rising Star

Thanks you so much @Artem Ervits

I will check and post result..

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

Mentor

Also make sure to open endpoints to outside for the ports you're using in azure portal.

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

@rama

Hive view will internally use: webhdfs file system uri So please check if the following property is set properly or not?

Click HDFS > Configs > Advanced > Advanced hdfs-site > dfs.namenode.http-address. When you enter the value in the view definition, pre-pend "webhdfs://" to the value you find in the advanced HDFS configuration settings.

For example, webhdfs://c6401.ambari.apache.org:50070

Please see: https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/section_get_v...