Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Recommendation for proxying Hadoop services without built-in Knox support

Solved Go to solution
Highlighted

Re: Recommendation for proxying Hadoop services without built-in Knox support

Expert Contributor

Thanks for sharing that! I followed your instructions and did the same for yarnui (adapting the paths slightly).

The root- and logs-redirection works, but many other redirections (especially those with {**} in the end) are not used by Knox.

Example: When calling https://172.18.10.163:8443/gateway/default/yarn, the site loads, but the static resources do not load. In /var/log/knox/gateway.log it says:

2016-05-12 11:13:34,109 DEBUG hadoop.gateway (GatewayFilter.java:doFilter(110)) - Received request: GET /yarn
2016-05-12 11:13:34,147 INFO  hadoop.gateway (KnoxLdapRealm.java:getUserDn(556)) - Computed userDn: uid=guest,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: guest
2016-05-12 11:13:34,227 INFO  hadoop.gateway (AclsAuthorizationFilter.java:init(62)) - Initializing AclsAuthz Provider for: YARNUI
2016-05-12 11:13:34,228 DEBUG hadoop.gateway (AclsAuthorizationFilter.java:init(70)) - ACL Processing Mode is: AND
2016-05-12 11:13:34,229 DEBUG hadoop.gateway (AclParser.java:parseAcls(59)) - No ACLs found for: YARNUI
2016-05-12 11:13:34,230 INFO  hadoop.gateway (AclsAuthorizationFilter.java:doFilter(85)) - Access Granted: true
2016-05-12 11:13:34,434 DEBUG hadoop.gateway (UrlRewriteProcessor.java:rewrite(155)) - Rewrote URL: https://172.18.10.163:8443/gateway/default/yarn, direction: IN via implicit rule: YARNUI/yarn/inbound/root to URL: http://resourcemanagerhost.local:8088/cluster
	[...]
2016-05-12 11:13:35,074 DEBUG hadoop.gateway (GatewayFilter.java:doFilter(110)) - Received request: GET /yarn/static/jquery/jquery-ui-1.9.1.custom.min.js
2016-05-12 11:13:35,417 DEBUG hadoop.gateway (GatewayFilter.java:doFilter(110)) - Received request: GET /yarn/static/jquery/jquery-1.8.2.min.js

That's the end of file. Nothing is logged after that.

I'm using HDP 2.3.4.7 with Knox 0.6.0.

I would appreciate your help, @Alex Miller or @Kevin Minder.

Thanks!

Re: Recommendation for proxying Hadoop services without built-in Knox support

@Benjamin R Does it work if you add a trailing slash?

Highlighted

Re: Recommendation for proxying Hadoop services without built-in Knox support

Expert Contributor

@Alex Miller This makes no difference. Still the page https://172.18.10.163:8443/gateway/default/yarn/ is loaded, but static resources or pages like https://172.18.10.163:8443/gateway/default/yarn/apps/ACCEPTED are not loaded.

edit: I found the error. In my topology file, I previously added a custom stanza (role. SERVICE-TEST) for which I created no service-definition. That made Knox behave weird. After removing that block, the YARN-UI over Knox works.

Thanks, Alex

Highlighted

Re: Recommendation for proxying Hadoop services without built-in Knox support

New Contributor

If you're looking to improve access to back-end service UI's for the ops team, as opposed to exposing the services to the larger user base, we make use of ssh tunneling via our admin jump hosts to effectively create personal SOCKS proxies for each ops/admin user.

We then use one of the dynamic proxy config plugins in Chrome or Firefox to direct requests to those services based on hostname, or in our case the domain of the hadoop environment.

This has the advantage of being very transparent and service URL's all tend to resolve correctly , including https based services. The disadvantage is that the person using this approach needs to know how to setup an ssh tunnel and how to configure their browser to use that tunnel for the Hadoop services.

Don't have an account?
Coming from Hortonworks? Activate your account here