question Re: Zeppelin + SparkR in Archives of Support Questions (Read Only)

Zeppelin + SparkR

gbraccialli3 — Thu, 05 Nov 2015 10:32:12 GMT

does zeppelin support SparkR interpreter?

Re: Zeppelin + SparkR

abajwa — Thu, 05 Nov 2015 10:35:09 GMT

Not yet. See this JIRA for more details:

https://issues.apache.org/jira/browse/ZEPPELIN-156

Re: Zeppelin + SparkR

nsabharwal — Thu, 05 Nov 2015 11:04:41 GMT

@Ali Bajwa

Jira is open but no timeline on support or updates. I wonder if there is a way to update jira to get some timelines.

Re: Zeppelin + SparkR

abajwa — Thu, 05 Nov 2015 11:11:57 GMT

Its on the roadmap for 0.6.0 apparently https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap

Re: Zeppelin + SparkR

nshawa — Thu, 05 Nov 2015 11:16:50 GMT

according to NFLABS this is coming any time (before end of the year)

Re: Zeppelin + SparkR

nsabharwal — Thu, 05 Nov 2015 20:40:17 GMT

@nshawa@hortonworks.com Is there a url that you can share ?

Re: Zeppelin + SparkR

abajwa — Fri, 06 Nov 2015 02:13:01 GMT

Actually the PR link in the jira above has the a lot more details. Sounds they have got it mostly working

Re: Zeppelin + SparkR

abajwa — Fri, 06 Nov 2015 02:13:32 GMT

See PR link mentioned in the JIRA

Re: Zeppelin + SparkR

nshawa — Fri, 27 Nov 2015 06:19:32 GMT

ok here is the latest, The R Interpreter for Zeppelin has not been merged yet with the latest Zeppelin dist. however you can use it now from here https://github.com/apache/incubator-zeppelin/pull/208. All the Best 🙂

Re: Zeppelin + SparkR

abajwa — Fri, 27 Nov 2015 06:23:39 GMT

I have been following that PR since the question was posted. Sounds like some serious discord between the contributor and community going on. Hopefully it doesn't impact the PR getting merged but doesn't look good.

Re: Zeppelin + SparkR

gbraccialli3 — Fri, 27 Nov 2015 07:33:48 GMT

As far as I understood this R interpreter PR is not sharing same SparkContext yet. I've already created a notebook that declares a function in Scala, another in Python and use SQL interpreter to call both functions in a single statement along with a custom java hive-udf. If we could add a function in R, it would be really nice.

Re: Zeppelin + SparkR

nsabharwal — Mon, 19 Aug 2019 12:52:45 GMT

@Guilherme Braccialli

check out this http://blog.sparkiq-labs.com/2015/11/16/interactive-data-science-with-r-in-apache-zeppelin-notebook/

Re: Zeppelin + SparkR

nsabharwal — Mon, 18 Jan 2016 21:13:53 GMT

@Guilherme Braccialli

https://community.hortonworks.com/articles/10365/apache-zeppelin-and-sparkr.html

Re: Zeppelin + SparkR

Hefei — Mon, 19 Aug 2019 12:52:37 GMT

Zeppelin since version 0.6, has provided support for R Interpreter. By default, the R Interpreter appears as two Zeppelin Interpreters, %r and %knitr. To run Zeppelin with the R Interpreter, some environment variables must be set:

R (3.0+)
JAVA_HOME (Oracle JDK 1.7+)
SPARK_HOME (The best way to do this is by editing conf/zeppelin-env.sh. If it is not set, the R Interpreter will not be able to interface with Spark.You should also copy conf/zeppelin-site.xml.template to conf/zeppelin-site.xml. That will ensure that Zeppelin sees the R Interpreter the first time it starts up.)

Then, clone the Zeppelin repository and build it with options:

>git clone https://github.com/apache/zeppelin.git 

# enable the "r" and "sparkr" profile 
>mvn clean install -e -DskipTests -Dspark.version={spark_version} -Dhadoop.version={hadoop_version} -Pr -Psparkr -Pvendor-repo -Pexamples -Drat.skip=true -Dcheckstyle.skip=true -Dcobertura.skip=true

And next, install SparkR package. In Spark 1.6 or earlier, the SparkR need to manually install.

>cd $SPARK_HOME 
>./R/install-dev.sh

Now, you can starting Apache Zeppelin with command line:

>cd $ZEPPELIN_HOME 
>bin/zeppelin-daemon.sh start  -e

After successful start, visit http://localhost:8080 with your web browser. And you can execute commands as in the CLI.

Re: Zeppelin + SparkR

aravind_kanda — Wed, 31 Aug 2016 06:25:31 GMT

What is the R environment variable that needs setup?