@Ameet Paranjape Can you double check you modified the zeppelin-env.sh as mentioned in the blog? The hdp.version should be replaced automatically without users having to change mapred-site configs (check note below about "message related to bad substitution") In the zeppelin-env.sh file, add the following.
Note: you will use PORT to access the Zeppelin Web UI. <HDP-version> corresponds to the version of HDP where you are installing Zeppelin; for example, 2.3.2.0-2950.
export HADOOP_CONF_DIR=/etc/hadoop/conf
export ZEPPELIN_PORT=9995
export ZEPPELIN_JAVA_OPTS="-Dhdp.version=<HDP-version>"
To obtain the HDP version for your HDP cluster, run the following command:
hdp-select status hadoop-client | sed 's/hadoop-client - \(.*\)/\1/' Add the following properties and settings:
spark.driver.extraJavaOptions -Dhdp.version=2.3.2.0-2950
spark.yarn.am.extraJavaOptions -Dhdp.version=2.3.2.0-2950
Note
Make sure that both spark.driver.extraJavaOptions & spark.yarn.am.extraJavaOptions are saved.
Without these properties set, the Spark job will fail with message related to bad substitution
Also note that the version of zeppelin used by in the TP blog is actually an early version of 0.5.5. I tested both this version as well as the released 0.5.5 zeppelin HDP 2.3.2 with both Spark 1.4.1 and Spark 1.5.1TP using the Zeppelin service and did not encounter issues running Spark code. The issues I encountered were around Hive (see here for details)
... View more
