Support Questions
Find answers, ask questions, and share your expertise

HDP 3.1.0 - Knox proxy HDFSUI - HTTP 401 Error

New Contributor

Hello,

I've recently deployed a HDP 3.1.0 cluster using Ambari on Ubuntu 18.04

RIght after deployment I've tried used the Knox UI admin interface to configure proxy-ing for various application web UIs.

All fine and dandy for Oozie, Yarn, WebHDFS, up until trying out HDFS UI which loads to a 500 Internal Error in the browser.

The topology file is configured as below for HDFS

<service>
<role>HDFSUI</role>
<url>http://namenode01:50070</url>
</service>

Now, I've tried setting the knox logs to debug, and I get the below entries in the 'gateway.log' log when attempting to connect, which yields the 401 HTTP error code.

2019-01-14 12:05:24,619 DEBUG knox.gateway (GatewayFilter.java:doFilter(119)) - Received request: GET /hdfs
2019-01-14 12:05:24,621 INFO  knox.gateway (KnoxLdapRealm.java:getUserDn(692)) - Computed userDn: uid=ldapuser,ou=people,dc=company,dc=com using dnTemplate for principal: ldapuser
2019-01-14 12:05:24,646 DEBUG knox.gateway (DefaultDispatch.java:executeOutboundRequest(121)) - Dispatch request: GET https://knoxgateway:443/gateway/bigdata/hdfs?user.name=ldapuser
2019-01-14 12:05:24,665 DEBUG knox.gateway (GatewayFilter.java:doFilter(119)) - Received request: GET /hdfs
2019-01-14 12:05:24,668 DEBUG knox.gateway (DefaultDispatch.java:executeOutboundRequest(134)) - Dispatch response status: 401
2019-01-14 12:05:24,668 DEBUG knox.gateway (DefaultDispatch.java:getInboundResponseContentType(209)) - Inbound response entity content type not provided.

I have tried setting the below parameters

hadoop.proxyuser.knox.groups=*

hadoop.proxyuser.knox.hosts=*

in

Ambari > hdfs > configs> core-site.xml

and restarting the cluster for changes to take effect, but no luck so far.

I do want to add that for this topology I have a remote OpenLDAP server setup, which does work in performing the auth via Knox for the other working components and as stated above, WebHDFS does work via Knox.

Any ideas are welcomed.

Thanks!

Matei C.

2 REPLIES 2

New Contributor

@Matei C What's the knox version? Maybe you can have a try with knox 0.13.0. I access HDFS File System successfully, but others for example "startupprocess" failed. That's because url is wrong. I'm making effort to solve the problem.

Hi @Matei C. this seems to be a known bug in Knox-1.0 used in HDP-3.1. One solution is to modify the URL used to access HDFS UI:

https://<knox-host>:8443/gateway/default/hdfs?host=http://<namenode-host>:50070 

Admittedly not a great solution but it worked for me. Another solution which I haven't tried is to insert the following line in the HDFSUI service definition in Knox topology file.

<version>2.7.0</version> 

This will use an older, bug-free version of HDFSUI service. More details here.