question Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway in Archives of Support Questions (Read Only)

Tutorial Issue: Can't connect to HDFS through the Knox Gateway

ads — Thu, 17 Dec 2015 10:42:02 GMT

I'm running though "Securing your Hadoop Infrastructure with Apache Knox." I've installed HDP 2.3 and have been following the steps described. I can run step 6, accessing the Hadoop Cluster via WebHDFS. But step 7, accessing the Hadopp Cluster via the Apache Know Gateway fails.

curl -iku guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'

HTTP/1.1 404 Not Found Cache-Control: must-revalidate,no-cache,no-store Content-Type: text/html;charset=ISO-8859-1 Content-Length: 1294 Server: Jetty(8.1.14.v20131031) <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 Not Found</title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /gateway/sandbox/webhdfs/v1/. Reason: <pre> Not Found</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html>

All the requisite processes seem to be running. I have no clue what to look at to get this working.

Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway

nsabharwal — Thu, 17 Dec 2015 10:49:57 GMT

@Christopher Lewis

what's the output of curl -iku guest:guest-password -X GET 'http://sandbox:50070/webhdfs/v1/?op=LISTSTATUS' ?

Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway

kevin_minder — Thu, 17 Dec 2015 10:52:36 GMT

If you've installed Knox via Ambari the topology name used is default instead of sandbox so try this command.

curl -iku guest:guest-password -X GET 'https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS'

Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway

ads — Thu, 17 Dec 2015 11:03:37 GMT

[root@sandbox ~]# curl -iku guest:guest-password -X GET 'http://sandbox:50070/webhdfs/v1/?op=LISTSTATUS' HTTP/1.1 200 OK Cache-Control: no-cache Expires: Thu, 17 Dec 2015 03:01:16 GMT Date: Thu, 17 Dec 2015 03:01:16 GMT Pragma: no-cache Expires: Thu, 17 Dec 2015 03:01:16 GMT Date: Thu, 17 Dec 2015 03:01:16 GMT Pragma: no-cache Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.26.hwx)

Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway

nsabharwal — Thu, 17 Dec 2015 11:06:11 GMT

@Christopher Lewis

Try this

curl -iv -k -u guest:guest-password https://sandbox:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS

Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway

ads — Thu, 17 Dec 2015 11:07:01 GMT

No, I am using the Knox included with the HDP image. If I try using <default> I get:

[root@sandbox ~]# curl -iku guest:guest-password -X GET 'https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS' HTTP/1.1 401 Unauthorized WWW-Authenticate: BASIC realm="application" Content-Length: 0 Server: Jetty(8.1.14.v20131031)

Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway

nsabharwal — Thu, 17 Dec 2015 11:07:33 GMT

@Kevin Minder Tutorial has errors

curl -iku guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'

http://hortonworks.com/hadoop-tutorial/securing-hadoop-infrastructure-apache-knox/

Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway

nsabharwal — Mon, 19 Aug 2019 12:32:07 GMT

It's working at my end.

Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway

kevin_minder — Thu, 17 Dec 2015 12:11:01 GMT

@Christopher Lewis < This looks like progress. I'm guessing the Demo LDAP server isn't running on your VM.

Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway

ads — Fri, 18 Dec 2015 00:59:17 GMT

Thank you both for your help. I restarted the Knox LDAP manually instead of with Ambari and used the default path instead of sandbox. I can get connected now.

@Neeraj Sabharwal

@Kevin Minder

Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway

beingmeus — Sat, 16 Jan 2016 07:32:58 GMT

It worked for me. Thanks for the advise

I wonder if HWX does this deliberately, so are we forced to venture into the dark space (so, we learn more) which we wouldn't otherwise 🙂

Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway

rkandula — Tue, 13 Dec 2016 09:48:46 GMT

Need to start Demo LDAP from service actions in order to test the test accounts with out LDAP integration

Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway

Amryz5 — Wed, 04 Apr 2018 18:19:39 GMT

how you restarted the Knox LDAP ???

Re: Tutorial Issue: Can't connect to HDFS through the Knox Gateway

v-weilwu — Fri, 28 Sep 2018 10:13:43 GMT

I'm having the same issue!

https://knox.apache.org/books/knox-1-1-0/user-guide.html#Quick+Start

Following the tutorial step by step and the I get this:

root@hn0-securi:~/knox-1.1.0# curl -i -k -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS' HTTP/1.1 500 Server Error Date: Thu, 27 Sep 2018 07:45:22 GMT Set-Cookie: JSESSIONID=5qbve5zzilryrp6nfy8zztn0;Path=/gateway/sandbox;Secure;HttpOnly Set-Cookie: rememberMe=deleteMe; Path=/gateway/sandbox; Max-Age=0; Expires=Wed, 26-Sep-2018 07:45:22 GMT Content-Type: text/html; charset=ISO-8859-1 Cache-Control: must-revalidate,no-cache,no-store Content-Length: 317 Connection: close Server: Jetty(9.2.22.v20170606) <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> <title>Error 500 Server Error</title> </head> <body><h2>HTTP ERROR 500</h2> <p>Problem accessing /gateway/sandbox/webhdfs/v1/. Reason: <pre> Server Error</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/> </body> </html>

Please help !!

Thanks