I've recently deployed a HDP 3.1.0 cluster using Ambari on Ubuntu 18.04
RIght after deployment I've tried used the Knox UI admin interface to configure proxy-ing for various application web UIs.
All fine and dandy for Oozie, Yarn, WebHDFS, up until trying out HDFS UI which loads to a 500 Internal Error in the browser.
The topology file is configured as below for HDFS
<service> <role>HDFSUI</role> <url>http://namenode01:50070</url> </service>
Now, I've tried setting the knox logs to debug, and I get the below entries in the 'gateway.log' log when attempting to connect, which yields the 401 HTTP error code.
2019-01-14 12:05:24,619 DEBUG knox.gateway (GatewayFilter.java:doFilter(119)) - Received request: GET /hdfs 2019-01-14 12:05:24,621 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(692)) - Computed userDn: uid=ldapuser,ou=people,dc=company,dc=com using dnTemplate for principal: ldapuser 2019-01-14 12:05:24,646 DEBUG knox.gateway (DefaultDispatch.java:executeOutboundRequest(121)) - Dispatch request: GET https://knoxgateway:443/gateway/bigdata/hdfs?user.name=ldapuser 2019-01-14 12:05:24,665 DEBUG knox.gateway (GatewayFilter.java:doFilter(119)) - Received request: GET /hdfs 2019-01-14 12:05:24,668 DEBUG knox.gateway (DefaultDispatch.java:executeOutboundRequest(134)) - Dispatch response status: 401 2019-01-14 12:05:24,668 DEBUG knox.gateway (DefaultDispatch.java:getInboundResponseContentType(209)) - Inbound response entity content type not provided.
I have tried setting the below parameters
Ambari > hdfs > configs> core-site.xml
and restarting the cluster for changes to take effect, but no luck so far.
I do want to add that for this topology I have a remote OpenLDAP server setup, which does work in performing the auth via Knox for the other working components and as stated above, WebHDFS does work via Knox.
Any ideas are welcomed.
@Matei C What's the knox version? Maybe you can have a try with knox 0.13.0. I access HDFS File System successfully, but others for example "startupprocess" failed. That's because url is wrong. I'm making effort to solve the problem.
Hi @Matei C. this seems to be a known bug in Knox-1.0 used in HDP-3.1. One solution is to modify the URL used to access HDFS UI:
Admittedly not a great solution but it worked for me. Another solution which I haven't tried is to insert the following line in the HDFSUI service definition in Knox topology file.
This will use an older, bug-free version of HDFSUI service. More details here.