Zeppelin R Interpreter in Master fails

wsalazar — Thu, 23 Jun 2016 02:51:39 GMT

After building Zeppelin 0.6.0 and I test the %spark.r %r interpreters and receive the error shown in https://issues.apache.org/jira/browse/ZEPPELIN-983

It seems to boil down to class not found for SparkRInterpreter

I use the most simple build process as it seems there is no need to use -Pr or Psparkr anymore

git clone https://github.com/apache/zeppelin
cd zeppelin
mvn clean package -DskipTests

from zeppelin-interpreter-spark-root-hostname.log

ERROR [2016-06-17 04:16:18,990] ({pool-1-thread-2} ProcessFunction.java[process]:41) - Internal error processing createInterpreter
org.apache.thrift.TException: java.lang.ClassNotFoundException: org.apache.zeppelin.spark.SparkRInterpreter
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.createInterpreter(RemoteInterpreterServer.java:175)
        at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$createInterpreter.getResult(RemoteInterpreterService.java:1409)
        at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$createInterpreter.getResult(RemoteInterpreterService.java:1394)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.zeppelin.spark.SparkRInterpreter
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:190)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.createInterpreter(RemoteInterpreterServer.java:148)
        ... 8 more

When I check the dependency jars I find no class SparkRInterpreter though it does exist in the src folder of ~/zeppelin/spark

[root@hostname zeppelin]# jar -tf spark/target/zeppelin-spark-0.6.0-SNAPSHOT.jar | grep SparkR


[root@hostname zeppelin]# ls spark/src/main/java/org/apache/zeppelin/spark/ | grep SparkR
SparkRInterpreter.java

Every other interpreter works as expected and can be found in the same dependency jar for Spark Interpreter

[root@rings hostname]# jar -tf spark/target/zeppelin-spark-0.6.0-SNAPSHOT.jar | grep PySpark
org/apache/zeppelin/spark/PySparkInterpreter.class


org/apache/zeppelin/spark/PySparkInterpreter$PythonInterpretRequest.class
[root@rings hostname]# jar -tf spark/target/zeppelin-spark-0.6.0-SNAPSHOT.jar | grep SparkSql
org/apache/zeppelin/spark/SparkSqlInterpreter.class


[root@rings hostname]# jar -tf spark/target/zeppelin-spark-0.6.0-SNAPSHOT.jar | grep DepInterpreter
org/apache/zeppelin/spark/DepInterpreter.class

Re: Zeppelin R Interpreter in Master fails

wsalazar — Thu, 23 Jun 2016 21:24:31 GMT

Found that the options -Pspark-X.X & -Psparkr are required for the build

Re: Zeppelin R Interpreter in Master fails

azeltov — Tue, 02 Aug 2016 02:51:04 GMT

@wsalazar This is what i use to build the zeppelin:

mvn clean package -Pspark-1.X -Dhadoop.version=2.X.0 -Phadoop-2.X -Pyarn -DskipTests

swap .X for the version you are building against.

It makes it YARN compatible.

question Re: Zeppelin R Interpreter in Master fails in Archives of Support Questions (Read Only)

Zeppelin R Interpreter in Master fails

Re: Zeppelin R Interpreter in Master fails

Re: Zeppelin R Interpreter in Master fails