Support Questions

Find answers, ask questions, and share your expertise
Celebrating as our community reaches 100,000 members! Thank you!

Recommendation for proxying Hadoop services without built-in Knox support


Knox 0.6.0 has built-in support for these 7 services:

  • WebHDFS
  • WebHCat
  • Oozie
  • HBase
  • Hive
  • Yarn
  • Storm

Is there a recommended approach to expose other services from the gateway host? Particularly web UIs, such as Ambari & Ranger.



These extensions are committed to the Apache Knox repo itself. They all use the config driven extension model so you need to look in the gateway-service-definitions module. In particular look in this directory. Now that you mention the openweathermap example, I need to update that to the new configuration based model at least as a comparison to the code based extension. The developers guide does briefly cover the config based extension.

View solution in original post


Expert Contributor

Thanks for sharing that! I followed your instructions and did the same for yarnui (adapting the paths slightly).

The root- and logs-redirection works, but many other redirections (especially those with {**} in the end) are not used by Knox.

Example: When calling, the site loads, but the static resources do not load. In /var/log/knox/gateway.log it says:

2016-05-12 11:13:34,109 DEBUG hadoop.gateway ( - Received request: GET /yarn
2016-05-12 11:13:34,147 INFO  hadoop.gateway ( - Computed userDn: uid=guest,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: guest
2016-05-12 11:13:34,227 INFO  hadoop.gateway ( - Initializing AclsAuthz Provider for: YARNUI
2016-05-12 11:13:34,228 DEBUG hadoop.gateway ( - ACL Processing Mode is: AND
2016-05-12 11:13:34,229 DEBUG hadoop.gateway ( - No ACLs found for: YARNUI
2016-05-12 11:13:34,230 INFO  hadoop.gateway ( - Access Granted: true
2016-05-12 11:13:34,434 DEBUG hadoop.gateway ( - Rewrote URL:, direction: IN via implicit rule: YARNUI/yarn/inbound/root to URL: http://resourcemanagerhost.local:8088/cluster
2016-05-12 11:13:35,074 DEBUG hadoop.gateway ( - Received request: GET /yarn/static/jquery/jquery-ui-1.9.1.custom.min.js
2016-05-12 11:13:35,417 DEBUG hadoop.gateway ( - Received request: GET /yarn/static/jquery/jquery-1.8.2.min.js

That's the end of file. Nothing is logged after that.

I'm using HDP with Knox 0.6.0.

I would appreciate your help, @Alex Miller or @Kevin Minder.



@Benjamin R Does it work if you add a trailing slash?

Expert Contributor

@Alex Miller This makes no difference. Still the page is loaded, but static resources or pages like are not loaded.

edit: I found the error. In my topology file, I previously added a custom stanza (role. SERVICE-TEST) for which I created no service-definition. That made Knox behave weird. After removing that block, the YARN-UI over Knox works.

Thanks, Alex


If you're looking to improve access to back-end service UI's for the ops team, as opposed to exposing the services to the larger user base, we make use of ssh tunneling via our admin jump hosts to effectively create personal SOCKS proxies for each ops/admin user.

We then use one of the dynamic proxy config plugins in Chrome or Firefox to direct requests to those services based on hostname, or in our case the domain of the hadoop environment.

This has the advantage of being very transparent and service URL's all tend to resolve correctly , including https based services. The disadvantage is that the person using this approach needs to know how to setup an ssh tunnel and how to configure their browser to use that tunnel for the Hadoop services.