Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to fix link errors in YARNUI with Knox?

How to fix link errors in YARNUI with Knox?

Expert Contributor

In HDP 2.6.5 manged by Ambari I have a working Knox topology exposing the Resource Manager Yarn UI with the folloing tags in the ui.xml (and Ambari managed default.xm) topology files:

  <service>
    <role>YARNUI</role>
    <url>http://{resourcemanager_server}:8088</url>
  </service>

  <service>
    <role>RESOURCEMANAGER</role>
    <url>http://{resourcemanager_server}:8088/ws</url>
  </service>

This is working and I'm able to access the YARN UI via Knox with the following URL:

https://knox.mydomain.org:8443/gateway/ui/yarn/

However when I access to the "ApplicationMaster" for a running Spark task, and I reach the Spark UI via the "/proxy/" subpath under the Resource Manager UI; the main page is shown correctly but most of the links under the "Jobs", "Stages" "SQL" and "JDBC/ODBC Server" (for Thift Server) are not working and give a Not Found error as shown.

96467-yarnui-spark-proxy-notfound.png

Based in the displayed error and comparing with the corresponding page when accessed directly in the Resource Manager UI (port 8088) it seems these links are missing the URL parameters (id and others) and this is the reason for the errors

How to fix this?

3 REPLIES 3
Highlighted

Re: How to fix link errors in YARNUI with Knox?

Expert Contributor

As with other Knox UI services (as HbaseUI or SparkHistoryUI) this error once again is related with some missing (and/or wrong) rules in the YARNUI service definitions instaled with HDP 2.6.5 under /var/lib/knox/data-2.6.5.0-292/services/yarnui/2.7.0/

To handle correctly these link URLs including arguments in the proxy pages patch the service's rewrite.xml file as follows:

--- rewrite.xml.hdp
+++ rewrite.xml@@ -35,6 +35,9 @@
 <rule dir="IN" name="YARNUI/yarn/inbound/cluster" pattern="*://*:*/**/yarn/cluster/{**}">
     <rewrite template="{$serviceUrl[YARNUI]}/cluster/{**}"/>
 </rule>
+<rule dir="IN" name="YARNUI/yarn/inbound/proxy1" pattern="*://*:*/**/yarn/proxy/{**}?{**}">
+    <rewrite template="{$serviceUrl[YARNUI]}/proxy/{**}?{**}"/>
+</rule>
 <rule dir="IN" name="YARNUI/yarn/inbound/proxy" pattern="*://*:*/**/yarn/proxy/{**}">
     <rewrite template="{$serviceUrl[YARNUI]}/proxy/{**}"/>
 </rule>
@@ -250,9 +253,13 @@
     <match pattern="*://*:*/proxy/{**}"/>
     <rewrite template="{$frontend[url]}/yarn/proxy/{**}"/>
 </rule>
-<rule dir="OUT" name="YARNUI/yarn/outbound/apps/history1">
-    <match pattern="/proxy/{**}"/>
-    <rewrite template="{$frontend[url]}/yarn/proxy/{**}"/>
+<rule dir="OUT" name="YARNUI/yarn/outbound/apps/history1" flow="OR">
+    <match pattern="/proxy/{**}?{**}">
+      <rewrite template="{$frontend[url]}/yarn/proxy/{**}?{**}"/>
+    </match>
+    <match pattern="/proxy/{**}">
+      <rewrite template="{$frontend[url]}/yarn/proxy/{**}"/>
+    </match>
 </rule>

Then remove the deployed runtime topology folders:

# rm -rf /usr/hdp/current/knox-server/data/deployments/*.topo.*

and the restart Knox and your links in the Knox/YarnUI proxied Spark UI will include the required parameters and work as expected:

96468-yarnui-spark-proxy-fixed.png

Re: How to fix link errors in YARNUI with Knox?

New Contributor

I'd love to provide myself with answers to my self questions too. My wish for 2019. ;)

Re: How to fix link errors in YARNUI with Knox?

Expert Contributor

@Isaac Arnault I invested (and spent) 6 hours trying to fix this and finally found a solutions. So because I'm still not able to create an article (because of the reputation system at this site) this was the only way I found to share this with the community to help others and avoid them to repeat the unsavory path ;)

My best wishes to you as well and I hope this may help you and someone else.

Best regards