- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Problem: Zeppelin 0.6 and Spark 2 on HDP 2.5
- Labels:
-
Apache Spark
-
Apache Zeppelin
Created on ‎11-23-2017 06:37 AM - edited ‎08-17-2019 09:28 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Dear experts,
I am running HDP 2.5 sandbox on Azure with Zeppelin 0.6.0.2.5 and Spark 2.0.0.2.5. Zeppelin is able to recognize only the Spark 1 but not 2, could you please help on this ? Are the versions incompatible ? I tried creating a new interpreter as spark2 as in attached images but still not working.
Below are the installed spark versions in my sandbox
[root@sandbox spark2]# spark-shell --version Multiple versions of Spark are installed but SPARK_MAJOR_VERSION is not set Spark1 will be picked by default Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 1.6.2 /_/ Type --help for more information. [root@sandbox spark2]# export SPARK_MAJOR_VERSION=2 [root@sandbox spark2]# spark-submit --version SPARK_MAJOR_VERSION is set to 2, using Spark2 Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 2.0.0.2.5.0.0-1245 /_/ Branch HEAD Compiled by user jenkins on 2016-08-26T02:52:23Z Revision c036d8e15666e914cd6aaf90ada0390619828b10 Url git@github.com:hortonworks/spark2.git Type --help for more information.
Created ‎11-23-2017 07:17 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Indeed HDP 2.5 Sandbox and Spark2 with Zeppelin are not compatible.
Read this Blog article for more infos:
https://de.hortonworks.com/blog/try-latest-innovations-apache-spark-apache-zeppelin-hortonworks-2-5-...
If you don't want to download the newer Sandbox version, try this tutorial:
https://community.hortonworks.com/articles/53029/how-to-install-and-run-spark-20-on-hdp-25-sandbox.h...
Created ‎11-23-2017 07:17 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Indeed HDP 2.5 Sandbox and Spark2 with Zeppelin are not compatible.
Read this Blog article for more infos:
https://de.hortonworks.com/blog/try-latest-innovations-apache-spark-apache-zeppelin-hortonworks-2-5-...
If you don't want to download the newer Sandbox version, try this tutorial:
https://community.hortonworks.com/articles/53029/how-to-install-and-run-spark-20-on-hdp-25-sandbox.h...
Created ‎11-23-2017 09:39 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks @Jan Dombrowicz ,i will use spark 1 with the zeppelin.
Created ‎11-25-2017 07:26 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
you can have a look at my last blog post, it may help you: Installing Apache Zeppelin 0.7.3 in HDP 2.5.3 with Spark and Spark2 Interpreters
Created ‎02-01-2018 10:54 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @Paul:
Thank you for the blog. I am working on HDP 2.5.0 in Azure cloud. I am on mac logged into sandbox from terminal in mac.
I am not able to locate the install directory of zeppelin. Can you give some pointers?
Thanks
Krishna
Created ‎02-01-2018 11:13 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I don't know exactly the detail of this Sandbox in the Azure Cloud.
Are you connecting to the Sandbox or to the docker container inside?
The docker container is where zeppelin and other services are located.
To connect to the docker container use the port 2222 in your SSH command.
Example: ssh root@127.0.0.1 -p 2222
It doesn't matter I guess if your cluster or sandbox is running on the cloud. You should be able to find zeppelin under /usr/hdp/current/zeppelin-server
Hope this helps.
BR. Paul
