question Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated. in Archives of Support Questions (Read Only)

java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

g_rao9704 — Sat, 05 Nov 2016 21:45:11 GMT

hd-hive.pngI am new to HDinsight hadoop cluster , when i have open and Ambari ->hive view>query and execute any hive query(show tables etc..) i am getting error: java.net:UnknowhostException: namenode

Note: I am able to trigger hive query in HIVE cli .

HDP 2.7

hive 1.2

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

aervits — Sun, 06 Nov 2016 00:07:56 GMT

in your Hive view, make sure webhdfs.url property is set to correct namenode url, I.e. webhdfs://fqdn:8020 or in case your cluster is HA, webhdfs://nameservices where nameservices is the dfs.nameservices property.

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

g_rao9704 — Sun, 06 Nov 2016 00:54:28 GMT

Thanks you so much@Artem Ervits

Can you help me find webhdfs.url property ..where i can find these property?

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

Former Member — Sun, 06 Nov 2016 01:48:49 GMT

@rama

Hive view will internally use: webhdfs file system uri So please check if the following property is set properly or not?

Click HDFS > Configs > Advanced > Advanced hdfs-site > dfs.namenode.http-address. When you enter the value in the view definition, pre-pend "webhdfs://" to the value you find in the advanced HDFS configuration settings.

For example, webhdfs://c6401.ambari.apache.org:50070

Please see: https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/section_get_values_non-standard_cluster_config.html

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

aervits — Sun, 06 Nov 2016 04:33:20 GMT

Look in hdfs-site.xml in advanced Ambari config for hdfs service

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

g_rao9704 — Sun, 06 Nov 2016 11:02:36 GMT

@Artem Ervits

sorry @Artem Ervits

Still i am unable to find the webhdfs.url property in hdfs_site.xml in hdfs service

I have attached hdfs services screen short please seehdfs-site1.png hdfs-site2.png hdfs-site3.png hdfs-site4.png

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

g_rao9704 — Sun, 06 Nov 2016 11:02:37 GMT

@Artem Ervits

sorry @Artem Ervits

Still i am unable to find the webhdfs.url property in hdfs_site.xml in hdfs service

I have attached hdfs services screen short please seehdfs-site1.png hdfs-site2.png hdfs-site3.png hdfs-site4.png

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

g_rao9704 — Sun, 06 Nov 2016 11:15:33 GMT

Thanks you@jss

I could not find dfs.namenode.http-address property too..i have share my hdfs_site screen short please help me to locate

advanced-hdfs.png

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

Former Member — Sun, 06 Nov 2016 12:00:36 GMT

@rama

Please use the search filter to find the property mentioned.

Search for "dfs.namenode.http-address" in the "Properties Filter" text box which is present in the

HDFS > Configs > Advanced (In this page) you will find a search box beside the "Manage Config Groups" just put the property "dfs.namenode.http-address" there to search.

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

g_rao9704 — Sun, 06 Nov 2016 12:27:02 GMT

Thanks you so much@jss

i could able to find the properties(attached screen short) ,so i need to edit properties ?hfds-http.png

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

Former Member — Sun, 06 Nov 2016 15:23:56 GMT

@rama

Hope the following article will help:

https://community.hortonworks.com/articles/49434/ambari-hive-view-webhdfs-filesystem-uri-in-hdfs-nn.html

You do not need to edit the "dfs.namenode.http-address' instead you need to refer to that value so that it can be used inside your Hive View.

Manage Ambai --> Views --> HiveView (check its property "WebHDFS FileSystem URI*") this value need to be changed.

When it comes to setting WebHDFS FileSystem URI* , make sure you set to webhdfs://${dfs.nameservices}   where ${dfs.nameservices} is the value defined in the "hdfs-site.xml"   With *NO* Port to be specified.  (When NameNode HA is configured)

Restart ambari-server

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

aervits — Sun, 06 Nov 2016 21:24:21 GMT

I should've realized that you're on Azure and using WASB, I am not 100 % sure but see if you can take the external address of the FS.defaultFS property and add to Hive view. You need to fill out the following properties in your Hive view with their associated values. This is in case it is HA, in case you also have kerberos, you need to add those properties as well. Keep in mind you need public address for each URL not the ones that says internal in their name. I have example REST API calls to create Ambari views in this script, properties in the curl calls are needed to setup views to function https://github.com/dbist/ambari-chef in that link go to recipes then ambari_views_setup.rb file for examples

"webhdfs.client.failover.proxy.provider" : "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider",      "webhdfs.ha.namenode.http-address.nn1" : "u1201.ambari.apache.org:50070",      "webhdfs.ha.namenode.http-address.nn2" : "u1201.ambari.apache.org:50070",      "webhdfs.ha.namenode.https-address.nn1" : null,      "webhdfs.ha.namenode.https-address.nn2" : null,      "webhdfs.ha.namenode.rpc-address.nn1" : "u1201.ambari.apache.org:8020",      "webhdfs.ha.namenode.rpc-address.nn2" : "u1202.ambari.apache.org:8020",      "webhdfs.ha.namenodes.list" : "nn1,nn2",      "webhdfs.nameservices" : "hacluster",      "webhdfs.url" : "webhdfs://hacluster",      "hive.host" : "u1203.ambari.apache.org",      "hive.http.path" : "cliservice",      "hive.http.port" : "10001",      "hive.metastore.warehouse.dir" : "/apps/hive/warehouse",      "hive.port" : "10000",      "hive.transport.mode" : "binary",      "yarn.ats.url" : "http://u1202.ambari.apache.org:8188",      "yarn.resourcemanager.url" : "u1202.ambari.apache.org:8088"

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

g_rao9704 — Sun, 06 Nov 2016 21:42:32 GMT

Thanks @jss

I will check and let you know the result..

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

g_rao9704 — Sun, 06 Nov 2016 21:43:26 GMT

Thanks you so much @Artem Ervits

I will check and post result..

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

aervits — Sun, 06 Nov 2016 21:50:08 GMT

Also make sure to open endpoints to outside for the ports you're using in azure portal.

Re: java.net:UnknownhostException:Namenode , While trigger hive job in HDinsight cluster,Can any one help me to out of this great appreciated.

g_rao9704 — Tue, 08 Nov 2016 16:58:44 GMT

HI @jss

I could find some configuration difference here

In PROD cluster ->managed ambari->views>hive->hive views->cluster configuration is set to "Local Ambari Managed Cluster" and its working fine

IN Dev cluster ->managed ambari->views>hive->hive views->cluster configuration is set to " custom" here it is not working

So should i change to "Local Ambari Managed Cluster"

if you want to me do this one , after save this should i need to restart the ambari server?

please suggest me