Created 06-12-2018 02:23 PM
Hi,
I've installed Apache Knox via Ambari.
Added Ambari service + configured Ldap.
Based on that, I was able to curl webhdfs as specific ldap use.
But when I access Ambari via knox in a Browser I can browse everything, but can't make changes to configuration.
Every time I change some configs in services I get this ajax response:
{ "status" : 400, "message" : "Invalid Request: Malformed Request Body. An exception occurred parsing the request body: Unexpected character ('%' (code 37)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: java.io.StringReader@3af8ede7; line: 1, column: 3]" }
When I access it directly, all works as expected.
Worth to mention, that I have nginx on top of knox (I need it for many reasons).
And my basic configuration is as follows:
server { listen 443 ssl; server_name gateway.example.com; ssl_certificate /etc/ssl/cert.crt; ssl_certificate_key /etc/ssl/cert.key; location / { proxy_pass https://knox:8443; # this magic is needed for WebSocket proxy_http_version 1.1; proxy_redirect off; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_set_header Host $http_host; } }
In my browser, I access Ambari like that:
https://gateway.example.com/gateway/default/ambari
Another thing that doesn't work for me regarding this matter is when I set this value:
default.app.topology.name to 'default' (as above url indicates).
I always get 500 error and some weird logs about rewriting.
Happy to provide more info, but not sure what else.
Created 06-12-2018 10:29 PM
While setting up knox for ambari we must define both the following roled inside the knos topology:
<service> <role>AMBARI</role> <url>http://$AMBARI_HOST:8080</url> </service> <service> <role>AMBARIUI</role> <url>http://$AMBARI_HOST:8080</url> </service>
.
After making the above changes please try to restart the knox and then try again.
Created 06-12-2018 02:35 PM
Okay, I've resolved that first issues (with config changes failures) by adding this to my topology:
<service> <role>AMBARI</role> <url>http://hostname:port</url> </service>
before I only had AMBARIUI service. Which now all makes sense.
But still, can't setup a default gateway so my urls are shorter.
Created 06-12-2018 02:54 PM
Very sorry to spam you with so many messages, but I'm attaching knox logs
==> /var/log/knox/gateway-audit.log <== 18/06/12 14:51:06 ||562e17ec-da4e-4c6f-8207-2e1ecfc149bb|audit|10.251.3.4|AMBARIUI||||access|uri|/gateway/default/ambari/ambari|unavailable|Request method: GET 18/06/12 14:51:06 ||562e17ec-da4e-4c6f-8207-2e1ecfc149bb|audit|10.251.3.4|AMBARIUI|anonymous|||authentication|uri|/gateway/default/ambari/ambari|success| ==> /var/log/knox/gateway.log <== 2018-06-12 14:51:06,681 INFO hadoop.gateway (AclsAuthorizationFilter.java:doFilter(85)) - Access Granted: true ==> /var/log/knox/gateway-audit.log <== 18/06/12 14:51:06 ||562e17ec-da4e-4c6f-8207-2e1ecfc149bb|audit|10.251.3.4|AMBARIUI|anonymous|||authorization|uri|/gateway/default/ambari/ambari|success| ==> /var/log/knox/gateway.log <== 2018-06-12 14:51:06,682 ERROR hadoop.gateway (UrlRewriteProcessor.java:rewrite(169)) - Failed to rewrite URL: https://gateway.example.com:443/ambari, direction: IN via rule: AMBARIUI/ambari/inbound/root, status: FAILUREexample.com 2018-06-12 14:51:06,682 ERROR hadoop.gateway (UrlRewriteProcessor.java:rewrite(169)) - Failed to rewrite URL: https://gateway.example.com:443/ambari, direction: IN via rule: AMBARIUI/ambari/inbound/root, status: FAILURE ==> /var/log/knox/gateway-audit.log <== 18/06/12 14:51:06 ||562e17ec-da4e-4c6f-8207-2e1ecfc149bb|audit|10.251.3.4|AMBARIUI|anonymous|||dispatch|uri||unavailable|Request method: GET 18/06/12 14:51:06 ||562e17ec-da4e-4c6f-8207-2e1ecfc149bb|audit|10.251.3.4|AMBARIUI|anonymous|||dispatch|uri||failure| ==> /var/log/knox/gateway.log <== 2018-06-12 14:51:06,684 WARN hadoop.gateway (DefaultDispatch.java:executeOutboundRequest(147)) - Connection exception dispatching request: org.apache.http.client.ClientProtocolException org.apache.http.client.ClientProtocolException at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) at org.apache.hadoop.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:130) at org.apache.hadoop.gateway.dispatch.DefaultDispatch.executeRequest(DefaultDispatch.java:116) at org.apache.hadoop.gateway.dispatch.DefaultDispatch.doGet(DefaultDispatch.java:278) at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:124) at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:107) at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.filter.AclsAuthorizationFilter.doFilter(AclsAuthorizationFilter.java:89) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:60) at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.filter.AnonymousAuthFilter$1.run(AnonymousAuthFilter.java:76) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.gateway.filter.AnonymousAuthFilter.continueWithEstablishedSecurityContext(AnonymousAuthFilter.java:71) at org.apache.hadoop.gateway.filter.AnonymousAuthFilter.doFilter(AnonymousAuthFilter.java:66) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:30) at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.GatewayFilter.doFilter(GatewayFilter.java:139) at org.apache.hadoop.gateway.GatewayFilter.doFilter(GatewayFilter.java:91) at org.apache.hadoop.gateway.GatewayServlet.service(GatewayServlet.java:141) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.apache.hadoop.gateway.filter.DefaultTopologyHandler.handle(DefaultTopologyHandler.java:84) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.http.ProtocolException: Target host is not specified at org.apache.http.impl.conn.DefaultRoutePlanner.determineRoute(DefaultRoutePlanner.java:71) at org.apache.http.impl.client.InternalHttpClient.determineRoute(InternalHttpClient.java:125) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ... 54 more 2018-06-12 14:51:06,685 ERROR hadoop.gateway (AbstractGatewayFilter.java:doFilter(63)) - Failed to execute filter: java.io.IOException: Service connectivity error. 2018-06-12 14:51:06,685 ERROR hadoop.gateway (AbstractGatewayFilter.java:doFilter(63)) - Failed to execute filter: java.io.IOException: Service connectivity error. 2018-06-12 14:51:06,685 ERROR hadoop.gateway (AbstractGatewayFilter.java:doFilter(63)) - Failed to execute filter: java.io.IOException: Service connectivity error. 2018-06-12 14:51:06,686 ERROR hadoop.gateway (GatewayFilter.java:doFilter(141)) - Gateway processing failed: java.io.IOException: Service connectivity error. java.io.IOException: Service connectivity error. at org.apache.hadoop.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:148) at org.apache.hadoop.gateway.dispatch.DefaultDispatch.executeRequest(DefaultDispatch.java:116) at org.apache.hadoop.gateway.dispatch.DefaultDispatch.doGet(DefaultDispatch.java:278) at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:124) at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:107) at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.filter.AclsAuthorizationFilter.doFilter(AclsAuthorizationFilter.java:89) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:60) at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.filter.AnonymousAuthFilter$1.run(AnonymousAuthFilter.java:76) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.gateway.filter.AnonymousAuthFilter.continueWithEstablishedSecurityContext(AnonymousAuthFilter.java:71) at org.apache.hadoop.gateway.filter.AnonymousAuthFilter.doFilter(AnonymousAuthFilter.java:66) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:30) at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.GatewayFilter.doFilter(GatewayFilter.java:139) at org.apache.hadoop.gateway.GatewayFilter.doFilter(GatewayFilter.java:91) at org.apache.hadoop.gateway.GatewayServlet.service(GatewayServlet.java:141) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.apache.hadoop.gateway.filter.DefaultTopologyHandler.handle(DefaultTopologyHandler.java:84) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:748) ==> /var/log/knox/gateway-audit.log <== 18/06/12 14:51:06 ||562e17ec-da4e-4c6f-8207-2e1ecfc149bb|audit|10.251.3.4|AMBARIUI|anonymous|||access|uri|/gateway/default/ambari/ambari|failure| ==> /var/log/knox/gateway.log <== 2018-06-12 14:51:06,686 ERROR hadoop.gateway (GatewayServlet.java:service(143)) - Gateway processing failed: java.io.IOException: Service connectivity error. java.io.IOException: Service connectivity error. at org.apache.hadoop.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:148) at org.apache.hadoop.gateway.dispatch.DefaultDispatch.executeRequest(DefaultDispatch.java:116) at org.apache.hadoop.gateway.dispatch.DefaultDispatch.doGet(DefaultDispatch.java:278) at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:124) at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:107) at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.filter.AclsAuthorizationFilter.doFilter(AclsAuthorizationFilter.java:89) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:60) at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.filter.AnonymousAuthFilter$1.run(AnonymousAuthFilter.java:76) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.gateway.filter.AnonymousAuthFilter.continueWithEstablishedSecurityContext(AnonymousAuthFilter.java:71) at org.apache.hadoop.gateway.filter.AnonymousAuthFilter.doFilter(AnonymousAuthFilter.java:66) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:30) at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332) at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232) at org.apache.hadoop.gateway.GatewayFilter.doFilter(GatewayFilter.java:139) at org.apache.hadoop.gateway.GatewayFilter.doFilter(GatewayFilter.java:91) at org.apache.hadoop.gateway.GatewayServlet.service(GatewayServlet.java:141) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.apache.hadoop.gateway.filter.DefaultTopologyHandler.handle(DefaultTopologyHandler.java:84) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:748)
Created 06-12-2018 10:29 PM
While setting up knox for ambari we must define both the following roled inside the knos topology:
<service> <role>AMBARI</role> <url>http://$AMBARI_HOST:8080</url> </service> <service> <role>AMBARIUI</role> <url>http://$AMBARI_HOST:8080</url> </service>
.
After making the above changes please try to restart the knox and then try again.
Created 06-13-2018 06:10 AM
Yes, the Ambari role was missing.
Do you know how to get rid of /gateway/default from the url based on the logs I provided above? Many thanks!