SparkR in Zepplin on HDP 2.5 sandbox Issues

[#1]

Hi

I'm trying to get the R Interperter on Zepplin working, as we need to prove that R can work in this environment for our analysists.

Am using sandbox version 2.5.0.0-1245

I have done the following on the sandbox:

yum install R R-devel libcurl-devel openssl-devel

And confirmed that R is working, and SparkR is working.

su spark
cd /usr/hdp/2.5.0.0-1245/current/share/bin 
./sparkR

In the console, I have started up livy and livy appears at the top of the interperter list.

Livy %livy, %pyspark, sparkr, %sql

livy.pyspark works but sparkr and spark do not. The error is "cannot start Sparkr"

%livy.sparkr 
hello <- function( name ) {
  sprintf( "Hello, %s", name ); 
} 
hello("livy")

Cannot start  sparkr.

What am I missing?

Thanks in advance

[#2]

Try restarting Livy interpreter, from Zeppelin's interpreter page, and check Livy's settings there. Also check livy's log in /var/log/zeppelin. No other specific ideas right now, it worked for me after doing roughly the same steps as you did (though on a real cluster, not on Sandbox).

[#3]

Hi

tried restarting everything, now have a "Internal SErver Error 500" here are the logs :

ERROR [2017-03-20 01:44:34,645] ({pool-2-thread-4} LivyHelper.java[createSession]:97) - sessionId:0.0 state is starting
ERROR [2017-03-20 01:44:35,658] ({pool-2-thread-4} LivyHelper.java[createSession]:97) - sessionId:0.0 state is starting
ERROR [2017-03-20 01:44:36,858] ({pool-2-thread-4} LivySparkRInterpreter.java[interpret]:91) - Exception in LivySparkRInterpreter while interpret
org.springframework.web.client.HttpServerErrorException: 500 Internal Server Error
        at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:94)
        at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:667)
        at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:620)
        at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:580)
        at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:498)
        at org.apache.zeppelin.livy.LivyHelper.executeHTTP(LivyHelper.java:358)
        at org.apache.zeppelin.livy.LivyHelper.executeCommand(LivyHelper.java:300)
        at org.apache.zeppelin.livy.LivyHelper.interpret(LivyHelper.java:241)
        at org.apache.zeppelin.livy.LivySparkRInterpreter.interpret(LivySparkRInterpreter.java:89)
        at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341)
        at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
        at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
 INFO [2017-03-20 01:44:36,875] ({pool-2-thread-4} SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1489974223298 finished by scheduler org.apache.zeppelin.livy.LivySparkRIn
terpreter285223730

[#4]

Hi

tried installing the full version, and am still getting the same error: this from the livy logs:

17/03/21 17:35:02 ERROR RSCClient: Failed to connect to context.
java.util.concurrent.TimeoutException: Timed out waiting for context to start.
        at com.cloudera.livy.rsc.ContextLauncher.connectTimeout(ContextLauncher.java:126)
        at com.cloudera.livy.rsc.ContextLauncher.access$100(ContextLauncher.java:62)
        at com.cloudera.livy.rsc.ContextLauncher$1.run(ContextLauncher.java:114)
        at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:123)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:380)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
        at java.lang.Thread.run(Thread.java:745)
17/03/21 17:35:02 INFO RSCClient: Failing pending job 22872eb6-951b-43aa-8df0-21dc9b4d5274 due to shutdown.
17/03/21 17:35:17 INFO RSCAppListener: Trying to stop app application_1490063492900_0005
17/03/21 17:35:17 ERROR RSCClient: Failed to connect to context.