Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

YARN ACL and Could not determine the proxy server for redirection exception for Spark UI.

Highlighted

YARN ACL and Could not determine the proxy server for redirection exception for Spark UI.

Explorer

Hi All,

 

 

After modifying yarn.admin.acl property from "*" to specific group, when I run spark2-shell from user which is not in this group and trying to access to Spark context Web UI I get error as below:

 

 

javax.servlet.ServletException: Could not determine the proxy server for redirection at org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.findRedirectUrl(AmIpFilter.java:201) at org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.doFilter(AmIpFilter.java:141) at org.spark_project.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.spark_project.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:461) at org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)

 

 

It works only when user is in yarn.admin.acl. I suspected that to see spark ui for own spark context it is not necessary to have yarn admin rights. Am' I right?

 

Thanks for any help.

 

 

Regards, Bart

Don't have an account?
Coming from Hortonworks? Activate your account here