Posts: 27
Registered: ‎11-05-2018

YARN ACL and Could not determine the proxy server for redirection exception for Spark UI.

Hi All,



After modifying yarn.admin.acl property from "*" to specific group, when I run spark2-shell from user which is not in this group and trying to access to Spark context Web UI I get error as below:



javax.servlet.ServletException: Could not determine the proxy server for redirection at org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.findRedirectUrl( at org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.doFilter( at org.spark_project.jetty.servlet.ServletHandler$CachedChain.doFilter( at org.spark_project.jetty.servlet.ServletHandler.doHandle( at org.spark_project.jetty.server.handler.ContextHandler.doHandle( at org.spark_project.jetty.servlet.ServletHandler.doScope( at org.spark_project.jetty.server.handler.ContextHandler.doScope( at org.spark_project.jetty.server.handler.ScopedHandler.handle( at org.spark_project.jetty.server.handler.gzip.GzipHandler.handle( at org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(



It works only when user is in yarn.admin.acl. I suspected that to see spark ui for own spark context it is not necessary to have yarn admin rights. Am' I right?


Thanks for any help.



Regards, Bart