question Re: Unable to connect to Spark History Server Web UI (Cloudera Enterprise 6.3.4) in Support Questions

Unable to connect to Spark History Server Web UI (Cloudera Enterprise 6.3.4)

Donal_RC — Thu, 06 Jan 2022 11:51:07 GMT

Accessing the URL on port 18088 returns a ERR_SSL_PROTOCOL_ERROR error in Chrome.

history server logs have the following:

11:25:29.130 AM WARN HttpParser Illegal character 0x16 in state=START for buffer HeapByteBuffer@1af6bd16[p=1,l=517,c=8192,r=516]={\x16<<< ...} 11:25:29.130 AM WARN HttpParser Illegal character 0x16 in state=START for buffer HeapByteBuffer@21158554[p=1,l=517,c=8192,r=516]={\x16<<< ...} 11:25:29.130 AM WARN HttpParser bad HTTP parsed: 400 Illegal character 0x16 for HttpChannelOverHttp@2390a367{r=0,c=false,a=IDLE,uri=null} 11:25:29.131 AM WARN HttpParser bad HTTP parsed: 400 Illegal character 0x16 for HttpChannelOverHttp@7843db5f{r=0,c=false,a=IDLE,uri=null}

Note: some lines truncated to avoid exposing data.

There was a similar post regarding Schema Registry UI in Ambari but the solution doesn't appear to be applicable here (though I suspect the issue is similar).

Any thoughts?

Re: Unable to connect to Spark History Server Web UI (Cloudera Enterprise 6.3.4)

Deepan_N — Thu, 06 Jan 2022 12:48:46 GMT

- Do we have enabled SSL for the Spark UI?

- If not, are we able to access the URL with the HTTP protocol? Also try with different browser and in Private Browsing mode.

- Have we tried accessing the URL with the CURL command?

- Do we have SPNEGO enabled for the cluster?

Re: Unable to connect to Spark History Server Web UI (Cloudera Enterprise 6.3.4)

mszurap — Thu, 06 Jan 2022 12:49:29 GMT

Hi @Donal_RC , Please check if the Spark History Server UI is TLS/SSL enabled or not. I suppose it is not, usually SHS WebUI is available only on http. (the log snippets suggest that it is plain http)

Chrome is automatically switching to https for some company domains, this feature is called HSTS - HTTP Strict Transport Security. Please check the addresss bar in Chrome, you should see that the URL starts with https.

If you have Internet Explorer, that doesn't do this, you can try there.

Alternatively you can:
- modify the URL and access the SHS with the host's IP address

- or enable TLS/SSL for the SHS too.

Hope this helps,

Miklos

Re: Unable to connect to Spark History Server Web UI (Cloudera Enterprise 6.3.4)

Donal_RC — Thu, 06 Jan 2022 13:51:51 GMT

This appears to be the cause. Switching to MS Edge avoided the HSTS feature in Chrome.

Will look at enabling HTTPS on all UIs though since that is the correct solution in the longterm (spark.ssl.historyServer.enabled option is currently set to false).