About douglasharley

douglasharley · ‎01-21-2019

Hi Sandeep, Thanks for your suggestion...I added all those properties to the Custom gateway-site section for Knox service in Ambari, with the same value I have specified for Zeppelin's zeppelin.websocket.max.text.message.size property (i.e., 3072000): Then I restarted Knox, and re-logged-in to Zeppelin and tried to open the Getting Started/Apache Spark in 5 Minutes notebook, but received the same errors in gateway.log: 2019-01-21 15:43:11,856 ERROR gateway.websockets (ProxyWebSocketAdapter.java:cleanupOnError(171)) - Error: org.eclipse.jetty.websocket.api.MessageTooLargeException: Resulting message size [73,728] is too large for configured max of [65,536] Did I not put those properties in the right place, or might I be missing something else? Thanks, Doug

douglasharley · ‎01-18-2019

Hello, I have setup Knox to proxy an HDP 3.1.0/HDF 3.3.0 cluster, but am having problems with Zeppelin...when I try to open a "large" notebook (no idea how big they actually are, since I cannot see them), like the Getting Started/Apache Spark in 5 Minutes notebook, Zeppelin hangs, and I see these errors in the Zeppelin logs: ({qtp1757676444-273} NotebookServer.java[onClose]:371) - Closed connection to 10.16.8.102 : 44880. (1006) EOF: Broken pipe The associated upstream exceptions in the Knox logs are: ERROR gateway.websockets (ProxyWebSocketAdapter.java:cleanupOnError(171)) - Error: org.eclipse.jetty.websocket.api.MessageTooLargeException: Resulting message size [73,728] is too large for configured max of [65,536] Does anyone know can this message size limitation be configured/overcome? I've been googling-around for an hour now and made zero progress. I saw this Knox issue, which was apparently fixed in the 0.12.0 release, but maybe not? https://issues.apache.org/jira/browse/KNOX-836 Thanks in advance for any assistance/advice... Cheers, Doug

douglasharley · ‎01-11-2019

You were exactly correct, thanks again! For others who might be using an NGINX proxy in front of Knox, the bolded below is what I had to add to me NGINX cfg for the proxy URL: server { listen 443 ssl; server_name PROXIED_URL; ssl on; ssl_protocols TLSv1.1 TLSv1.2; ssl_certificate /etc/ssl/private/CORPORATE_CERT.crt; ssl_certificate_key /etc/ssl/private/CORPORATE_CERT.key; location / { proxy_pass https://KNOX_HOST:8443; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_read_timeout 86400; } }

douglasharley · ‎01-09-2019

Aha, that might just be it... I do have NGINX in front of Knox, but just proxying it, not load-balancing. I bet you are right, and if I configure NGINX for web sockets it will work. I just got thrown-off my those Knox errors and didn't even think to check NGINX logs, herpderp! Yes, the role is ZEPPELINWS. I am in the middle of swapping-out the self-signed certs for corporate ones, but once I get that taken care of will do the NGINX cfg changes and if works will accept your answer. THANKS for the excellent reply!!! Cheers, Doug

douglasharley · ‎01-08-2019

I am attempting to configure Zeppelin behind Knox gateway running within an HDP 3.1.0 cluster with HDF 3.3.1 installed, all administered by Ambari 2.7.3. I am following the docs here: https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.1.0/configuring-proxy-knox/sec_configuring_proxy_knox.pdf. As per the docs, I have set gateway.websocket.feature.enabled=true in Ambari's Knox configs, and when I access Zeppelin at the proxied URL I get the UI, saying the websocket is disconnected, and in the knox logs I see this: 2019-01-08 20:26:46,217 DEBUG knox.gateway (GatewayFilter.java:doFilter(119)) - Received request: GET /zeppelin/ws 2019-01-08 20:26:46,217 DEBUG knox.gateway (UrlRewriteProcessor.java:rewrite(163)) - Rewrote URL: https://KNOXHOST:8443/gateway/CLUSTERNAME/zeppelin/ws, direction: IN via explicit rule: ZEPPELINWS/zeppelin/ws/inbound to URL: ws://ZEPPELINHOST:9995/ws/ws 2019-01-08 20:26:46,218 DEBUG knox.gateway (DefaultDispatch.java:executeOutboundRequest(121)) - Dispatch request: GET ws://ZEPPELINHOST:9995/ws/ws 2019-01-08 20:26:46,219 WARN knox.gateway (DefaultDispatch.java:executeOutboundRequest(147)) - Connection exception dispatching request: ws://ZEPPELINHOST:9995/ws/ws org.apache.http.conn.UnsupportedSchemeException: ws protocol is not supported Seeing the extra "ws" appended to the redirect URL, I add a pre-shortened URL to empty ZEPPELINWS service definition in the knox descriptor, to override the bad one: ws://ZEPPELINHOST:9995 Then save descriptor/redeploy the topology and also restart knox, then I see essentially the same error, but with the correct URL now in the knox logs: 2019-01-08 20:41:12,796 DEBUG knox.gateway (GatewayFilter.java:doFilter(119)) - Received request: GET /zeppelin/ws 2019-01-08 20:41:12,798 DEBUG knox.gateway (UrlRewriteProcessor.java:rewrite(163)) - Rewrote URL: https://KNOXHOST:8443/gateway/CLUSTERNAME/zeppelin/ws, direction: IN via explicit rule: ZEPPELINWS/zeppelin/ws/inbound to URL: ws://ZEPPELINHOST:9995/ws 2019-01-08 20:41:12,800 DEBUG knox.gateway (DefaultDispatch.java:executeOutboundRequest(121)) - Dispatch request: GET ws://ZEPPELINHOST:9995/ws 2019-01-08 20:41:12,802 WARN knox.gateway (DefaultDispatch.java:executeOutboundRequest(147)) - Connection exception dispatching request: ws://ZEPPELINHOST:9995/ws org.apache.http.conn.UnsupportedSchemeException: ws protocol is not supported Any ideas what my problem might be? I have been banging my head on this for almost a full day and not making any progress, so thanks in advance for any assistance... -Doug

douglasharley · ‎01-02-2019

OK, I figured this out...the problem was with the default setting of knoxsso.token.ttl property, which was 30000 (30 seconds). I stopped the Knox service, updated the configuration in Ambari, restarted Knox, and now behavior is correct.

douglasharley · ‎01-02-2019

Hello, I have used Ambari 2.7.3 to setup an HDP 3.1.0 + HDF 3.3.0 cluster, and am trying to configure Knox proxy functionality but am having problems...the Knox admin UI doesn't seem to respect the setting of 30 minute sessions with the ShiroProvider. I have verified that the configs in Ambari all specify sessionTimeout of 30, which is supposed to be 30 minutes, but I get logged-out almost immediately when trying to use the admin UI. If I click down into the default read-only topologies, and then try to navigate back into the providers, I have to re-login, even though it's on;y been like 30 seconds. Any idea what the problem might be? Thanks in advance, Doug

douglasharley · ‎09-20-2018

Nevermind, used the PDF docs (https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.2.0/installing-hdf/hdf-installing-hdf.pdf) with new VMs and all is good now.

douglasharley · ‎09-15-2018

@Akhil S Naik This still appears to be a problem...I am attempting to install HDF 3.2.0.0 into Ambari 2.7.0.0 that is managing a brand new HDP 3.0.0 cluster, management pack installed fine, but when I ayyempt to specify the version info file URL (had to google for it: http://public-repo-1.hortonworks.com/HDF/ubuntu16/3.x/updates/3.2.0.0/HDF-3.2.0.0-520.xml) Ambari complains: "An internal system exception occurred: Stack data, Stack HDF 3.2 is not found in Ambari metainfo" Any idea what might be the issue(s)? Thanks, Doug

Online	Offline
Last Visited	‎01-31-2019 02:39 AM

Member Since	‎08-24-2018 02:20 PM
Last Visited	‎01-31-2019 02:39 AM
Posts	16
Kudos received	1

Cloudera Community

Cloudera Community

Re: Knox admin UI session length not respected wit...

Re: Zeppelin websocket message size limitations wh...

Zeppelin websocket message size limitations when p...

Re: Knox + Zeppelin websocket problems

Re: Knox + Zeppelin websocket problems

Knox + Zeppelin websocket problems

Re: Knox admin UI session length not respected wit...

Knox admin UI session length not respected with de...

Re: Installing HDF on Ambari with HDP 3.0.0.0 fail...

Re: Installing HDF on Ambari with HDP 3.0.0.0 fail...