Apache Zeppelin Tech Preview Live

[#1]

http://hortonworks.com/hadoop-tutorial/apache-zeppelin/

The Zeppelin TP is built against Spark 1.4.1 in HDP. We are also about to publish Spark 1.5.1 TP very soon and once that is out Zeppelin TP will also be revised to carry instructions for Spark 1.5.1.

Please play with it and post here if you run into any issues.

[#2]

Looks good. Installation was simple.

Blog needs minor editing.

Add the following properties and settings:

spark.driver.extraJavaOptions -Dhdp.version=2.3.2.0-2950 spark.yarn.am.extraJavaOptions -Dhdp.version=2.3.2.0-2950

this should be

spark.driver.extraJavaOptions -Dhdp.version=2.3.2.0-2950 

spark.yarn.am.extraJavaOptions -Dhdp.version=2.3.2.0-2950

For newbies, we may want to share this in the blog as it works like charm

[#3]

Check samples notebook section for a link to a few notebooks.

[#4]

Installed this through the Ambari service for testing and basic Spark, SparkSQL, PySpark seem ok

Couple of issues:

1. tried out the Magellan blog notebook (after modifying it to include the %dep from the blog) and the UberRecord cell errors out:

From the log:

5/10/24 10:48:06 INFO SchedulerFactory: Job remoteInterpretJob_1445708886505 started by scheduler org.apache.zeppelin.spark.SparkInterpreter313266037
15/10/24 10:48:06 ERROR Job: Job failed
scala.reflect.internal.Types$TypeError: bad symbolic reference. A signature in Shape.class refers to term geometry
in value com.core which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when compiling Shape.class.
 at scala.reflect.internal.pickling.UnPickler$Scan.toTypeError(UnPickler.scala:847)
 at scala.reflect.internal.pickling.UnPickler$Scan$LazyTypeRef.complete(UnPickler.scala:854)
 at scala.reflect.internal.pickling.UnPickler$Scan$LazyTypeRef.load(UnPickler.scala:863)
 at scala.reflect.internal.Symbols$Symbol.typeParams(Symbols.scala:1489)
 at scala.tools.nsc.transform.SpecializeTypes$$anonfun$scala$tools$nsc$transform$SpecializeTypes$$normalizeMember$1.apply(SpecializeTypes.scala:798)
 at scala.tools.nsc.transform.SpecializeTypes$$anonfun$scala$tools$nsc$transform$SpecializeTypes$$normalizeMember$1.apply(SpecializeTypes.scala:798)
 at scala.reflect.internal.SymbolTable.atPhase(SymbolTable.scala:207)
 at scala.reflect.internal.SymbolTable.beforePhase(SymbolTable.scala:215)
 at scala.tools.nsc.transform.SpecializeTypes.scala$tools$nsc$transform$SpecializeTypes$$norma

(side note: this notebook doesn't seem to have much documentation on what its doing like the other...would be good to add)

2. The blog currently says the below

This technical preview can be installed on any HDP 2.3.x cluster

...however 2.3.0 comes with Spark 1.3.1 which will not work unless they manually install Spark 1.4.1 TP so either:

a) we may want to include steps for those users too (esp since the current version of the sandbox comes with 1.3.1)

b) explicitly ask users to try the Zeppelin TP with 2.3.2

[#5]

Zeppelin Ambari service has been updated to install the updated TP Zeppelin bits for Spark 1.4.1 and 1.3.1. The update will be made for 1.5.1 this week after the TP is out

Also the Magellan notebook has also been updated with documentation and to enable it to run standalone on 1.4.1

[#6 · Accepted solution]

Added a couple of sentences of clarification to get folks to Spark 1.4.1.

If you have an HDP 2.3.0 cluster, it came with Spark 1.3.1, you can either upgrade the entire cluster withAmbari to 2.3.2 to get Spark 1.4.1 or only manually upgrade Spark to 1.4.1

[#7]

Tried this on a 2.3.2 cluster (brand new build) with 1.4.1, and had the same problem with Zeppelin and Magellan. Seems like Zeppelin is doing something to the context.