Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Accessing hadoop through webhdfs

Accessing hadoop through webhdfs

New Contributor

I am using HDP , i would like to use the webhdfs api of hadoop through .Net hadoop webhdfs api, i would like to access the hadoop in a remote machine through the webhdfs , i would like to know the URI that should be done to aceess hdp remotely using http.It should be something like http://host:port/webhdfs, what is the port to use Thank you

4 REPLIES 4
Highlighted

Re: Accessing hadoop through webhdfs

Re: Accessing hadoop through webhdfs

@Maher Hattabi

The default port is 50070. To get a list of files in a directory you would use:

curl -i "http://sandbox.hortonworks.com:50070/webhdfs/v1/user/root/output/?op=LISTSTATUS"

Re: Accessing hadoop through webhdfs

New Contributor

@Michael Young thanks for your replay ,am developping a .Net app, in order to access a remote hdfs cluster here is the code i used when it is locallly i got a correct answer when i put the ip of the remote cluster i got an exception List<string> lstDirectoriesName = new List<string>(); try { WebHDFSClient hdfsClient = new WebHDFSClient(new Uri("http://io-dell-svr8:50070"), "Administrator"); Microsoft.Hadoop.WebHDFS.DirectoryListing directroyStatus = hdfsClient.GetDirectoryStatus("/dataset").Result; List<DirectoryEntry> lst = directroyStatus.Files.ToList(); foreach (DirectoryEntry var in lst) { lstDirectoriesName.Add(var.PathSuffix); } return lstDirectoriesName; } catch (Exception exException) { Console.WriteLine(exException.Message); return null; }

Any help whould be appreciated

Re: Accessing hadoop through webhdfs

Expert Contributor

@Maher Hattabi,

If it works when connected to your local sandbox, but fails when you try to go to io-dell-svr8:50070, you need to see if your computer can correctly resolve the name io-dell-svr8 and make sure there is no firewall blocking the connection to 50070 on that host.

From a cmd prompt, you should be able to 'nslookup io-dell-svr8' to see if the name resolves. Telnet is not installed by default on windows 10, but if you have putty, you can try to connect to port 50070 and see if it connects. If it times out, you have firewalls between you that you will have to have opened.

I hope this helps!

John