@David_Ghazaleh, Did the response assist in resolving your query? If it did, please mark the relevant reply as the solution, as it will help others locate the answer more easily in the future. ... View more

Hello @Boron  I believe you are using HDP 3.x. Note that there is no Spark 1.x available in HDP 3. We need to use Spark 2.x. Set the SPARK_HOME to Spark 2. export SPARK_HOME=/usr/hdp/current/spark2-client ... View more

hi @Deepan_N    by running the command below directly in python3:   r0.headers["www-authenticate"]   returns the following error:   Python 3.6.8 (default, Nov 16 2020, 16:55:22) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> r0.headers["www-authenticate"] Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'r0' is not defined >>> below is the screenshot of the commands executed in bash:   ... View more

Hi @loridigia    If cluster/application is not enabled dynamic allocation and if you set --conf spark.executor.instances=1 then it will launch only 1 executor. Apart from executor, you will see AM/driver in the Executor tab Spark UI. ... View more

This appears to be the cause. Switching to MS Edge avoided the HSTS feature in Chrome.   Will look at enabling HTTPS on all UIs though since that is the correct solution in the longterm (spark.ssl.historyServer.enabled option is currently set to false). ... View more