Support Questions

Find answers, ask questions, and share your expertise

SparkR in Zepplin on HDP 2.5 sandbox Issues

New Contributor


I'm trying to get the R Interperter on Zepplin working, as we need to prove that R can work in this environment for our analysists.

Am using sandbox version

I have done the following on the sandbox:

yum install R R-devel libcurl-devel openssl-devel

And confirmed that R is working, and SparkR is working.

su spark
cd /usr/hdp/ 

In the console, I have started up livy and livy appears at the top of the interperter list.

Livy %livy, %pyspark, sparkr, %sql

livy.pyspark works but sparkr and spark do not. The error is "cannot start Sparkr"

hello <- function( name ) {
  sprintf( "Hello, %s", name ); 

Cannot start  sparkr.

What am I missing?

Thanks in advance


Try restarting Livy interpreter, from Zeppelin's interpreter page, and check Livy's settings there. Also check livy's log in /var/log/zeppelin. No other specific ideas right now, it worked for me after doing roughly the same steps as you did (though on a real cluster, not on Sandbox).

New Contributor


tried restarting everything, now have a "Internal SErver Error 500" here are the logs :

ERROR [2017-03-20 01:44:34,645] ({pool-2-thread-4}[createSession]:97) - sessionId:0.0 state is starting
ERROR [2017-03-20 01:44:35,658] ({pool-2-thread-4}[createSession]:97) - sessionId:0.0 state is starting
ERROR [2017-03-20 01:44:36,858] ({pool-2-thread-4}[interpret]:91) - Exception in LivySparkRInterpreter while interpret
org.springframework.web.client.HttpServerErrorException: 500 Internal Server Error
        at org.springframework.web.client.DefaultResponseErrorHandler.handleError(
        at org.springframework.web.client.RestTemplate.handleResponse(
        at org.springframework.web.client.RestTemplate.doExecute(
        at org.springframework.web.client.RestTemplate.execute(
        at org.apache.zeppelin.livy.LivyHelper.executeHTTP(
        at org.apache.zeppelin.livy.LivyHelper.executeCommand(
        at org.apache.zeppelin.livy.LivyHelper.interpret(
        at org.apache.zeppelin.livy.LivySparkRInterpreter.interpret(
        at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(
        at org.apache.zeppelin.scheduler.FIFOScheduler$
        at java.util.concurrent.Executors$
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(
        at java.util.concurrent.ScheduledThreadPoolExecutor$
        at java.util.concurrent.ThreadPoolExecutor.runWorker(
        at java.util.concurrent.ThreadPoolExecutor$
 INFO [2017-03-20 01:44:36,875] ({pool-2-thread-4}[jobFinished]:137) - Job remoteInterpretJob_1489974223298 finished by scheduler org.apache.zeppelin.livy.LivySparkRIn

New Contributor


tried installing the full version, and am still getting the same error: this from the livy logs:

17/03/21 17:35:02 ERROR RSCClient: Failed to connect to context.
java.util.concurrent.TimeoutException: Timed out waiting for context to start.
        at com.cloudera.livy.rsc.ContextLauncher.connectTimeout(
        at com.cloudera.livy.rsc.ContextLauncher.access$100(
        at com.cloudera.livy.rsc.ContextLauncher$
        at io.netty.util.concurrent.PromiseTask$
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(
        at io.netty.util.concurrent.SingleThreadEventExecutor$
17/03/21 17:35:02 INFO RSCClient: Failing pending job 22872eb6-951b-43aa-8df0-21dc9b4d5274 due to shutdown.
17/03/21 17:35:17 INFO RSCAppListener: Trying to stop app application_1490063492900_0005
17/03/21 17:35:17 ERROR RSCClient: Failed to connect to context.