Accessing hadoop through webhdfs

[#1]

I am using HDP , i would like to use the webhdfs api of hadoop through .Net hadoop webhdfs api, i would like to access the hadoop in a remote machine through the webhdfs , i would like to know the URI that should be done to aceess hdp remotely using http.It should be something like http://host:port/webhdfs, what is the port to use Thank you

[#2]

@Maher Hattabi

Please check link below -

https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/WebHDFS.html

For examples also refer -

https://dzone.com/articles/hadoop-rest-api-webhdfs

https://bighadoop.wordpress.com/2013/06/02/hadoop-rest-api-webhdfs/

[#3]

@Maher Hattabi

The default port is 50070. To get a list of files in a directory you would use:

curl -i "http://sandbox.hortonworks.com:50070/webhdfs/v1/user/root/output/?op=LISTSTATUS"

[#4]

@Michael Young thanks for your replay ,am developping a .Net app, in order to access a remote hdfs cluster here is the code i used when it is locallly i got a correct answer when i put the ip of the remote cluster i got an exception List<string> lstDirectoriesName = new List<string>(); try { WebHDFSClient hdfsClient = new WebHDFSClient(new Uri("http://io-dell-svr8:50070"), "Administrator"); Microsoft.Hadoop.WebHDFS.DirectoryListing directroyStatus = hdfsClient.GetDirectoryStatus("/dataset").Result; List<DirectoryEntry> lst = directroyStatus.Files.ToList(); foreach (DirectoryEntry var in lst) { lstDirectoriesName.Add(var.PathSuffix); } return lstDirectoriesName; } catch (Exception exException) { Console.WriteLine(exException.Message); return null; }

Any help whould be appreciated

[#5]

@Maher Hattabi,

If it works when connected to your local sandbox, but fails when you try to go to io-dell-svr8:50070, you need to see if your computer can correctly resolve the name io-dell-svr8 and make sure there is no firewall blocking the connection to 50070 on that host.

From a cmd prompt, you should be able to 'nslookup io-dell-svr8' to see if the name resolves. Telnet is not installed by default on windows 10, but if you have putty, you can try to connect to port 50070 and see if it connects. If it times out, you have firewalls between you that you will have to have opened.

I hope this helps!

John