Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

WebHDFS to download files via KNOX in C#

WebHDFS to download files via KNOX in C#

New Contributor

Hi all

I'm been teraring my hair for the last week or so trying to download files from HDFS in a C# web app via KNOX in our corporate intranet. I am using WebRequest. Listing files works just fine with WebRequest:

request.Credentials = new NetworkCredential(user, pass);

url:

https://<host>:8443/gateway/default/webhdfs/v1/data/system/STP/sound_files/2017-09-19/?op=LISTSTATUS

But as soon as I try to access a file with url:

https://<host>:8443/gateway/default/webhdfs/v1/data/system/STP/sound_files/2017-09-19/filename.wav?o... get 401 Unautherized.

I have tried everything I can imagine. Fiddling with Credentials, using WebClient and DownloadFile, I have tried:

string authInfo = user + ":" + pass;

authInfo = Convert.ToBase64String(Encoding.Default.GetBytes(authInfo));

req.Headers["Authorization"] = "Basic " + authInfo;

I have tried the &noredirect=true in the url to see if I get the return URL from the service but no, same problem.

I have tried

CredentialCache myCredenalCache = new CredentialCache();

myCredenalCache.Add(<host>, 443, "Basic", new NetworkCredential(user, pass));

I'm about to give up!

I know that my user has access because I can download the files in a browser no problem. It's when I run it in the app, I get problems.

Anyone?

Don't have an account?
Coming from Hortonworks? Activate your account here