Support Questions

Find answers, ask questions, and share your expertise

Tutorial Issue: Can't connect to HDFS through the Knox Gateway

avatar
Explorer

I'm running though "Securing your Hadoop Infrastructure with Apache Knox." I've installed HDP 2.3 and have been following the steps described. I can run step 6, accessing the Hadoop Cluster via WebHDFS. But step 7, accessing the Hadopp Cluster via the Apache Know Gateway fails.

curl -iku guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'

HTTP/1.1 404 Not Found Cache-Control: must-revalidate,no-cache,no-store Content-Type: text/html;charset=ISO-8859-1 Content-Length: 1294 Server: Jetty(8.1.14.v20131031) <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 Not Found</title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /gateway/sandbox/webhdfs/v1/. Reason: <pre> Not Found</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html>

All the requisite processes seem to be running. I have no clue what to look at to get this working.

1 ACCEPTED SOLUTION

avatar
Master Mentor
13 REPLIES 13

avatar
Master Mentor

@Christopher Lewis

what's the output of curl -iku guest:guest-password -X GET 'http://sandbox:50070/webhdfs/v1/?op=LISTSTATUS' ?

avatar

If you've installed Knox via Ambari the topology name used is default instead of sandbox so try this command.

curl -iku guest:guest-password -X GET 'https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS'

avatar
Explorer

No, I am using the Knox included with the HDP image. If I try using <default> I get:

[root@sandbox ~]# curl -iku guest:guest-password -X GET 'https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS' HTTP/1.1 401 Unauthorized WWW-Authenticate: BASIC realm="application" Content-Length: 0 Server: Jetty(8.1.14.v20131031)

avatar
Master Mentor

891-screen-shot-2015-12-16-at-101821-pm.png

It's working at my end.

avatar

@Christopher Lewis < This looks like progress. I'm guessing the Demo LDAP server isn't running on your VM.

avatar
Master Mentor

@Kevin Minder Tutorial has errors

curl -iku guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'

http://hortonworks.com/hadoop-tutorial/securing-hadoop-infrastructure-apache-knox/

avatar
New Contributor

It worked for me. Thanks for the advise

I wonder if HWX does this deliberately, so are we forced to venture into the dark space (so, we learn more) which we wouldn't otherwise 🙂

avatar
Explorer

[root@sandbox ~]# curl -iku guest:guest-password -X GET 'http://sandbox:50070/webhdfs/v1/?op=LISTSTATUS' HTTP/1.1 200 OK Cache-Control: no-cache Expires: Thu, 17 Dec 2015 03:01:16 GMT Date: Thu, 17 Dec 2015 03:01:16 GMT Pragma: no-cache Expires: Thu, 17 Dec 2015 03:01:16 GMT Date: Thu, 17 Dec 2015 03:01:16 GMT Pragma: no-cache Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26.hwx)

<then the JSON output>

avatar
Master Mentor