Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Apache Knox - 404 error on accessing hdfs using WebHDFS

Solved Go to solution

Apache Knox - 404 error on accessing hdfs using WebHDFS

Expert Contributor

Hi all,

I'm trying to secure HDP using Apache Knox .. using the HDP Apache Knox tutorial,

and running into following issues, when firing the following commands ->

curl -iku guest:guest-password -X put 'https://sandbox.hortonworks.com:8443/gateway/knox_sample/webhdfs/v1/user/guest/knox-sample?op=MKDIRS&permission=777'

Any ideas on this ?

[root@sandbox knox-server]# curl -iku guest:guest-password -X put 'https://sandbox.hortonworks.com:8443/gateway/knox_sample/webhdfs/v1/user/guest/knox-sample?op=MKDIRS&permission=777' HTTP/1.1 404 Not Found Date: Thu, 12 Jan 2017 06:24:17 GMT Cache-Control: must-revalidate,no-cache,no-store Content-Type: text/html; charset=ISO-8859-1 Content-Length: 331 Server: Jetty(9.2.15.v20160210) <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /gateway/knox_sample/webhdfs/v1/user/guest/knox-sample. Reason: <pre> Not Found</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html>

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Apache Knox - 404 error on accessing hdfs using WebHDFS

Rising Star

Hi @Karan Alang , you might be having multiple topology files under '/etc/knox/conf/topologies/' , you can have multiple topology files under that location in the form <topology_name>.xml, and knox will automatically pick up these new topology definition files.

Then the URL for each service (curl call) will change from /gateway/default/HIVE to /gateway/<topologu_name>/HIVE for the topology named <topology_name>.xml

May be, you only have configured default.xml topology file and hence its working.

NOTE : These custom topology files need to be managed and maintained outside of Ambari, since Ambari only manages the default topology. Because of this, any changes made to the default topology via Ambari will need to be manually propagated to the customized topology files.

Hope this helps !

4 REPLIES 4

Re: Apache Knox - 404 error on accessing hdfs using WebHDFS

Expert Contributor
  1. curl -iku guest:guest-password -X put 'https://sandbox.hortonworks.com:8443/gateway/default/webhdfs/v1/user/guest/knox-sample?op=MKDIRS&permission=777'

i made the change highlighted, and i was able to make it work i.e. the gateway/default instead of gateway/knox_sample

Not sure if that is the correct fix, do i need to rename the default Knox gateway (to knox_sample) ?

i'm new to Knox, so appreciate your responses.

Re: Apache Knox - 404 error on accessing hdfs using WebHDFS

Expert Contributor

@Neeraj Sabharwal, @Sunile Manjee - any ideas on this ?

Highlighted

Re: Apache Knox - 404 error on accessing hdfs using WebHDFS

Rising Star

Hi @Karan Alang , you might be having multiple topology files under '/etc/knox/conf/topologies/' , you can have multiple topology files under that location in the form <topology_name>.xml, and knox will automatically pick up these new topology definition files.

Then the URL for each service (curl call) will change from /gateway/default/HIVE to /gateway/<topologu_name>/HIVE for the topology named <topology_name>.xml

May be, you only have configured default.xml topology file and hence its working.

NOTE : These custom topology files need to be managed and maintained outside of Ambari, since Ambari only manages the default topology. Because of this, any changes made to the default topology via Ambari will need to be manually propagated to the customized topology files.

Hope this helps !

Re: Apache Knox - 404 error on accessing hdfs using WebHDFS

Contributor

The gateway/knox_sample assumes a topology named knox_sample.xml in your {GATEWAY_HOME}/conf/topologies directory. If that doesn't exist then you will get a 404.

As mentioned by others, if you are using Ambari to make the changes then you are using the default.xml topology since that is the only topology that Ambari is aware.