Created on
02-04-2020
09:58 PM
- last edited on
02-05-2020
12:54 AM
by
VidyaSargur
I have an HA hadoop cluster with 2 namenodes and the nameserviceId as "nameservice1" . When I am trying to make connection to this cluster using url like below it gives me error :
curl -i "http://nameservice1/webhdfs/v1/user/hdfs/?blocksize=67108864&replication=3&permission=744&dfs.nameservices=nameservice1&dfs.ha.namenodes.nameservice1=master1,master2&dfs.namenode.http-address.nameservice1.master1=master1.hadoop.com:50070&dfs.namenode.http-address.nameservice1.master2=master2.hadoop.com:50070&dfs.namenode.rpc-address.nameservice1.master1=master1.hadoop.com:8022&dfs.namenode.rpc-address.nameservice1.master2=master2.hadoop.com:8022&dfs.defaultFS=hdfs://nameservice1&user=hdfs&op=LISTSTATUS"
Ho can one call an HA hadoop cluster using webhdfs?
Created on 02-04-2020 10:14 PM - edited 02-04-2020 10:20 PM
WebHDFS does not natively support Namenode HA (means you can not use "namesrvice" name in the URL) but you can use Knox to provide the functionality.
Like https://cwiki.apache.org/confluence/display/KNOX/Examples+WebHDFS
SO either you can configure components like Knox in front of nameNodes Or you will have to use the webhdfs with individual NameNode address:port in the webhdfs URL.