Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to reference an HA hadoop cluster using the nameserviceId?

how to reference an HA hadoop cluster using the nameserviceId?

New Contributor

I have an HA hadoop cluster with 2 namenodes and the nameserviceId as "nameservice1" . When I am trying to make connection to this cluster using url like below it gives me error :

 

curl -i "http://nameservice1/webhdfs/v1/user/hdfs/?blocksize=67108864&replication=3&permission=744&dfs.nameservices=nameservice1&dfs.ha.namenodes.nameservice1=master1,master2&dfs.namenode.http-address.nameservice1.master1=master1.hadoop.com:50070&dfs.namenode.http-address.nameservice1.master2=master2.hadoop.com:50070&dfs.namenode.rpc-address.nameservice1.master1=master1.hadoop.com:8022&dfs.namenode.rpc-address.nameservice1.master2=master2.hadoop.com:8022&dfs.defaultFS=hdfs://nameservice1&user=hdfs&op=LISTSTATUS"

 

Ho can one call an HA hadoop cluster using webhdfs?

1 REPLY 1
Highlighted

Re: how to reference an HA hadoop cluster using the nameserviceId?

Super Mentor

@bigdataC 

 

WebHDFS does not natively support Namenode HA (means you can not use "namesrvice" name in the URL) but you can use Knox to provide the functionality.

Like https://cwiki.apache.org/confluence/display/KNOX/Examples+WebHDFS

.
https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/configuring-proxy-knox/content/ha_configure_we...

 

SO either you can configure components like Knox in front of nameNodes  Or you will have to use the webhdfs with individual NameNode address:port in the webhdfs URL.

Don't have an account?
Coming from Hortonworks? Activate your account here