Recommendation for proxying Hadoop services without built-in Knox support

[#1]

Knox 0.6.0 has built-in support for these 7 services:

• WebHDFS
• WebHCat
• Oozie
• HBase
• Hive
• Yarn
• Storm

Is there a recommended approach to expose other services from the gateway host? Particularly web UIs, such as Ambari & Ranger.

[#2]

Thanks for sharing that! I followed your instructions and did the same for yarnui (adapting the paths slightly).

The root- and logs-redirection works, but many other redirections (especially those with {**} in the end) are not used by Knox.

Example: When calling https://172.18.10.163:8443/gateway/default/yarn, the site loads, but the static resources do not load. In /var/log/knox/gateway.log it says:

2016-05-12 11:13:34,109 DEBUG hadoop.gateway (GatewayFilter.java:doFilter(110)) - Received request: GET /yarn
2016-05-12 11:13:34,147 INFO  hadoop.gateway (KnoxLdapRealm.java:getUserDn(556)) - Computed userDn: uid=guest,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: guest
2016-05-12 11:13:34,227 INFO  hadoop.gateway (AclsAuthorizationFilter.java:init(62)) - Initializing AclsAuthz Provider for: YARNUI
2016-05-12 11:13:34,228 DEBUG hadoop.gateway (AclsAuthorizationFilter.java:init(70)) - ACL Processing Mode is: AND
2016-05-12 11:13:34,229 DEBUG hadoop.gateway (AclParser.java:parseAcls(59)) - No ACLs found for: YARNUI
2016-05-12 11:13:34,230 INFO  hadoop.gateway (AclsAuthorizationFilter.java:doFilter(85)) - Access Granted: true
2016-05-12 11:13:34,434 DEBUG hadoop.gateway (UrlRewriteProcessor.java:rewrite(155)) - Rewrote URL: https://172.18.10.163:8443/gateway/default/yarn, direction: IN via implicit rule: YARNUI/yarn/inbound/root to URL: http://resourcemanagerhost.local:8088/cluster
	[...]
2016-05-12 11:13:35,074 DEBUG hadoop.gateway (GatewayFilter.java:doFilter(110)) - Received request: GET /yarn/static/jquery/jquery-ui-1.9.1.custom.min.js
2016-05-12 11:13:35,417 DEBUG hadoop.gateway (GatewayFilter.java:doFilter(110)) - Received request: GET /yarn/static/jquery/jquery-1.8.2.min.js

That's the end of file. Nothing is logged after that.

I'm using HDP 2.3.4.7 with Knox 0.6.0.

I would appreciate your help, @Alex Miller or @Kevin Minder.

Thanks!

[#3]

@Benjamin R Does it work if you add a trailing slash?

[#4]

@Alex Miller This makes no difference. Still the page https://172.18.10.163:8443/gateway/default/yarn/ is loaded, but static resources or pages like https://172.18.10.163:8443/gateway/default/yarn/apps/ACCEPTED are not loaded.

edit: I found the error. In my topology file, I previously added a custom stanza (role. SERVICE-TEST) for which I created no service-definition. That made Knox behave weird. After removing that block, the YARN-UI over Knox works.

Thanks, Alex

[#5]

If you're looking to improve access to back-end service UI's for the ops team, as opposed to exposing the services to the larger user base, we make use of ssh tunneling via our admin jump hosts to effectively create personal SOCKS proxies for each ops/admin user.

We then use one of the dynamic proxy config plugins in Chrome or Firefox to direct requests to those services based on hostname, or in our case the domain of the hadoop environment.

This has the advantage of being very transparent and service URL's all tend to resolve correctly , including https based services. The disadvantage is that the person using this approach needs to know how to setup an ssh tunnel and how to configure their browser to use that tunnel for the Hadoop services.