Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: The Cloudera Community will undergo maintenance on Saturday, August 17 at 12:00am PDT. See more info here.

YARN ACL and Could not determine the proxy server for redirection exception for Spark UI.

YARN ACL and Could not determine the proxy server for redirection exception for Spark UI.

Contributor

Hi All,

 

 

After modifying yarn.admin.acl property from "*" to specific group, when I run spark2-shell from user which is not in this group and trying to access to Spark context Web UI I get error as below:

 

 

javax.servlet.ServletException: Could not determine the proxy server for redirection at org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.findRedirectUrl(AmIpFilter.java:201) at org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.doFilter(AmIpFilter.java:141) at org.spark_project.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.spark_project.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:461) at org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)

 

 

It works only when user is in yarn.admin.acl. I suspected that to see spark ui for own spark context it is not necessary to have yarn admin rights. Am' I right?

 

Thanks for any help.

 

 

Regards, Bart