We use Jupyterhub with our HDP cluster. When starting a PySpark kernel we have the problem the the link to the Spark UI is not working. When following the link we get an HTTP ERROR 500 with the following detail:
javax.servlet.ServletException: Could not determine the proxy server for redirection
The Spark application itself is running and the Spark UI can also be accessed by following the link in the Resourcemanager UI.
But Amabri does not allow me to remove the old settings related to non-HA. It says that this field is required. Is that normal behavior? The YARN documentation for HA says that the old settings must be replaced with the ones related to HA.