question Hue Notebook how do I switch between spark versions I am using livy,Hue on HDP 2.5 Livy not switching spark version from 1.6 to 2.0 in Archives of Support Questions (Read Only)

Hue Notebook how do I switch between spark versions I am using livy,Hue on HDP 2.5 Livy not switching spark version from 1.6 to 2.0

stephen1 — Tue, 21 Apr 2026 13:47:33 GMT

Hi I am trying to run spark jobs using hue notebook but everything runs in spark 1.6 how can I switch or select spark 2. I am running on HDP 2.5. Hope its not a silly question and thanks for any help.

Hi I am trying to use hue notebook on HDP2.5 to run spark jobs and I want to run them in spark 2 but it seems to stay stuck in 1.6 I am not sure how to switch or specify that I would like to run my test code in spark 2. Hopefully this is not a silly question. Super thanks for any help.

Re: Hue Notebook how do I switch between spark versions I am using livy,Hue on HDP 2.5 Livy not switching spark version from 1.6 to 2.0

jhorsch — Wed, 22 Feb 2017 03:01:18 GMT

Livy 0.2 does not work with Spark 2 that I have found. Upgrading to Livy 0.3 and modifying the livy-env.sh to:

export SPARK_HOME=/usr/hdp/current/spark2-client

export SPARK_MAJOR_VERSION=2

This also works for Zeppelin 0.7. With livy 0.2 Zeppelin was throwing a 404 error attempting to get the version from livy.

Re: Hue Notebook how do I switch between spark versions I am using livy,Hue on HDP 2.5 Livy not switching spark version from 1.6 to 2.0

stephen1 — Wed, 22 Feb 2017 19:17:32 GMT

Hi jhorsch, thank you so much for taking the time to answer. You are an absolute superstar!!!! It worked a charm just perfect! Our Data Scientist says she owes you a beer!!!

Super thanks

Stephen