Support Questions

Find answers, ask questions, and share your expertise

Spark 1.6 with Zeppelin

avatar
Contributor

Hi,

Is there any Zeppelin Tech Preview which is made to work with Spark 1.6 on HDP? I have setup Zeppelin 0.6.0 with Spark 1.6 But looks its not yet supported.

Thanks

Vijay

1 ACCEPTED SOLUTION

avatar

Zeppelin ships with Spark embedded with it. If you follow the Zeppelin tech preview you can get it to work with Spark 1.5.x in HDP 2.3.4.

In Apache, Zeppelin released a version compatible with Spark 1.6 on Jan 22nd. You can get the binary of that Zeppelin from https://zeppelin.incubator.apache.org/download.html and follow the Zeppelin TP http://hortonworks.com/hadoop-tutorial/apache-zeppelin/ to get it work with Spark 1.6 Tech Preview (http://hortonworks.com/hadoop-tutorial/apache-spark-1-6-technical-preview-with-hdp-2-3/)

View solution in original post

6 REPLIES 6

avatar
Master Mentor

avatar
Master Mentor

avatar
Contributor

Artem,

I have setup both Spark and Zeppelin following the above links. However, when I create a zeppelin notebook and run sc.version, I get a 'Spark 1.6 is not supprted message'. So wondering whether there is any other alternative to get this working. Please see the below screenshot. Thanks

1625-zeppelin-copy.png

avatar
Master Mentor

@vbhoomireddy both zepplin and Spark 1.6 are technical preview so yes we do not officially support it until its stable very soon.

avatar

Zeppelin ships with Spark embedded with it. If you follow the Zeppelin tech preview you can get it to work with Spark 1.5.x in HDP 2.3.4.

In Apache, Zeppelin released a version compatible with Spark 1.6 on Jan 22nd. You can get the binary of that Zeppelin from https://zeppelin.incubator.apache.org/download.html and follow the Zeppelin TP http://hortonworks.com/hadoop-tutorial/apache-zeppelin/ to get it work with Spark 1.6 Tech Preview (http://hortonworks.com/hadoop-tutorial/apache-spark-1-6-technical-preview-with-hdp-2-3/)

avatar
Contributor

Thanks @vshukla Timely news! Got it working now. Kind regards