Tutorial Issue: Can't connect to HDFS through the Knox Gateway

[#1]

I'm running though "Securing your Hadoop Infrastructure with Apache Knox." I've installed HDP 2.3 and have been following the steps described. I can run step 6, accessing the Hadoop Cluster via WebHDFS. But step 7, accessing the Hadopp Cluster via the Apache Know Gateway fails.

curl -iku guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'

HTTP/1.1 404 Not Found Cache-Control: must-revalidate,no-cache,no-store Content-Type: text/html;charset=ISO-8859-1 Content-Length: 1294 Server: Jetty(8.1.14.v20131031) <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 Not Found</title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /gateway/sandbox/webhdfs/v1/. Reason: <pre> Not Found</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html>

All the requisite processes seem to be running. I have no clue what to look at to get this working.

[#2]

@Christopher Lewis

what's the output of curl -iku guest:guest-password -X GET 'http://sandbox:50070/webhdfs/v1/?op=LISTSTATUS' ?

[#3]

If you've installed Knox via Ambari the topology name used is default instead of sandbox so try this command.

curl -iku guest:guest-password -X GET 'https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS'

[#4]

No, I am using the Knox included with the HDP image. If I try using <default> I get:

[root@sandbox ~]# curl -iku guest:guest-password -X GET 'https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS' HTTP/1.1 401 Unauthorized WWW-Authenticate: BASIC realm="application" Content-Length: 0 Server: Jetty(8.1.14.v20131031)

[#5]

It's working at my end.

[#6]

@Christopher Lewis < This looks like progress. I'm guessing the Demo LDAP server isn't running on your VM.

[#7]

@Kevin Minder Tutorial has errors

curl -iku guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'

http://hortonworks.com/hadoop-tutorial/securing-hadoop-infrastructure-apache-knox/

[#8]

It worked for me. Thanks for the advise

I wonder if HWX does this deliberately, so are we forced to venture into the dark space (so, we learn more) which we wouldn't otherwise 🙂

[#9]

[root@sandbox ~]# curl -iku guest:guest-password -X GET 'http://sandbox:50070/webhdfs/v1/?op=LISTSTATUS' HTTP/1.1 200 OK Cache-Control: no-cache Expires: Thu, 17 Dec 2015 03:01:16 GMT Date: Thu, 17 Dec 2015 03:01:16 GMT Pragma: no-cache Expires: Thu, 17 Dec 2015 03:01:16 GMT Date: Thu, 17 Dec 2015 03:01:16 GMT Pragma: no-cache Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26.hwx)

<then the JSON output>

[#10 · Accepted solution]

@Christopher Lewis

Try this

curl -iv -k -u guest:guest-password https://sandbox:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS