question Re: Spark 2 and Zeppelin in Archives of Support Questions (Read Only)

Spark 2 and Zeppelin

ymoiseev — Wed, 05 Oct 2016 12:28:28 GMT

Hi,

I was wondering if anyone tried setting up Zeppelin to work with Spark2 on HDP 2.5? As far as I understand, I can switch Spark version via SPARK_MAJOR_VERSION variable, and it works for spark-submit, but Zeppelin is still using Spark1.

Ideally, I want to have two different interpreter groups for every major Spark version, but just just switching it to Spark2 will be a good start. Any ideas?

Re: Spark 2 and Zeppelin

pankaj_singh — Wed, 05 Oct 2016 13:49:09 GMT

There is a related defect https://issues.apache.org/jira/browse/ZEPPELIN-1390 which has fix version as 0.6.1 and 0.7.0

Re: Spark 2 and Zeppelin

sunile_manjee — Thu, 06 Oct 2016 02:55:32 GMT

@vshukla Has this been backported into 0.6.0 which is version of zeppelin in hdp 2.5?

Re: Spark 2 and Zeppelin

vshukla — Thu, 06 Oct 2016 03:08:32 GMT

Zeppelin in HDP 2.5 does not support Spark 2. A technical preview of this will land in Hortonworks Data Cloud in next month or so.