Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to expose Spark History UI with Knox in HDP 2.6.5?

Highlighted

How to expose Spark History UI with Knox in HDP 2.6.5?

Expert Contributor

I needed to expose the Spark History with Knox in HDP 2.6.5 and find out this is not included by Ambari in the default topology. I found some old references on this forum which may be valid for older versions, but refer to a service definition named "SPARKUI" which is not present with Knox 0.12.0 installed in HDP 2.6.5:

https://community.hortonworks.com/questions/32872/knox-wrong-mapping-help-will-be-appriciated.html

By looking under "/usr/hdp/current/knox-server/data/services" I found out the Spark History UI service definition at "sparkhistoryui/1.4.0/service.xml" and this showed me the current name for the Knox service is "SPARKHISTORYUI".

So I included this in my topology definition:

Knox_SparkHistoryUI_Blank_Screenshot  <service>
    <role>SPARKHISTORYUI</role>
    <url><spark_history_host>:18081</url>
  </service>

Then when I access the Knox URL "https://knox.datalake.net:8443/gateway/default/sparkhistory/" I reach to the Spark History UI, but most of the page is blank and Firefox Inspector I get a lot of "404 Not Found" errors as shown in the attached screenshot.

Based on the logs this seems to be related with some missing and/or wrong pattern definitions in the rewrite.xml file for the service. So what are the needed fixes and configurations to make Spark History UI work with Knox in HDP 2.6.5??

96453-knox-sparkhistoryui-blank-screenshot.jpg

2 REPLIES 2

Re: How to expose Spark History UI with Knox in HDP 2.6.5?

Expert Contributor

First of all if you want the quick solution look at this:

https://community.hortonworks.com/questions/88485/knox-with-spark-2x-history-ui-not-working-works-wi...

I managed to solve this my own way, before finding the previous response; so I'm keeping my soluiton in case the information is useful to someone else facing the same problem.

The original "/sparkhistoryui/1.4.0/rewrite.xml" has an extensive but incomplete list of rewrite rules for files under "/static/*" subfolder, starting with these at line 41. Following the example in the rewrite.xml from the working "HDFSUI" service I replaced all these with the single "catch-all" rule:

  <rule dir="OUT" name="SPARKHISTORYUI/sparkhistory/outbound/static" pattern="/static/{**}">
    <rewrite template="{$frontend[url]}/sparkhistory/static/{**}"/>
  </rule>

After this all the "404 Not Found" under static where solver, but I still have an error for the following URL:

https://knox.datacenterantel.uy:8443/api/v1/applications?limit=2147483647

I tried to solve this by adding a rule for the "/api/v1/applications/{**}" pattern but this didn't work

Searching the web I found some comments in a bug report (JIRA KNOX-1160) related to Knox and the SparkHistoryUI service plugin affecting different version of Spark 2.x. The suggested fix for Spark versions <= 2.3.0 was to include the following property (depending on your topology name) in the "spark-defaults.conf" file (" Spark2 >> Config >> Custom spark2-defaults" in Ambari):

spark.ui.proxyBase=/gateway/default/sparkhistory

After doing this and restarting the Spark service all the "404 Non Found" errors are gone and the Service History UI seems to be working as expected.

I hope this helps to someone else and thanks to @Marco Gaido for the previous response (I saw after posting this question/answer).

Re: How to expose Spark History UI with Knox in HDP 2.6.5?

New Contributor

This way of changing configuration is working for me. But the issue is now cluster URL(http://<HOST>:18080/) is facing same issue.

Please let us know what to do to make both cluster URL and KNOX URL to work.