Created 03-19-2017 11:03 PM
Hi
I'm trying to get the R Interperter on Zepplin working, as we need to prove that R can work in this environment for our analysists.
Am using sandbox version 2.5.0.0-1245
I have done the following on the sandbox:
yum install R R-devel libcurl-devel openssl-devel
And confirmed that R is working, and SparkR is working.
su spark cd /usr/hdp/2.5.0.0-1245/current/share/bin ./sparkR
In the console, I have started up livy and livy appears at the top of the interperter list.
Livy %livy, %pyspark, sparkr, %sql
livy.pyspark works but sparkr and spark do not. The error is "cannot start Sparkr"
%livy.sparkr hello <- function( name ) { sprintf( "Hello, %s", name ); } hello("livy") Cannot start sparkr.
What am I missing?
Thanks in advance
Created 03-20-2017 12:30 AM
Try restarting Livy interpreter, from Zeppelin's interpreter page, and check Livy's settings there. Also check livy's log in /var/log/zeppelin. No other specific ideas right now, it worked for me after doing roughly the same steps as you did (though on a real cluster, not on Sandbox).
Created 03-20-2017 02:18 AM
Hi
tried restarting everything, now have a "Internal SErver Error 500" here are the logs :
ERROR [2017-03-20 01:44:34,645] ({pool-2-thread-4} LivyHelper.java[createSession]:97) - sessionId:0.0 state is starting ERROR [2017-03-20 01:44:35,658] ({pool-2-thread-4} LivyHelper.java[createSession]:97) - sessionId:0.0 state is starting ERROR [2017-03-20 01:44:36,858] ({pool-2-thread-4} LivySparkRInterpreter.java[interpret]:91) - Exception in LivySparkRInterpreter while interpret org.springframework.web.client.HttpServerErrorException: 500 Internal Server Error at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:94) at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:667) at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:620) at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:580) at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:498) at org.apache.zeppelin.livy.LivyHelper.executeHTTP(LivyHelper.java:358) at org.apache.zeppelin.livy.LivyHelper.executeCommand(LivyHelper.java:300) at org.apache.zeppelin.livy.LivyHelper.interpret(LivyHelper.java:241) at org.apache.zeppelin.livy.LivySparkRInterpreter.interpret(LivySparkRInterpreter.java:89) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341) at org.apache.zeppelin.scheduler.Job.run(Job.java:176) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) INFO [2017-03-20 01:44:36,875] ({pool-2-thread-4} SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1489974223298 finished by scheduler org.apache.zeppelin.livy.LivySparkRIn terpreter285223730
Created 03-21-2017 04:44 AM
Hi
tried installing the full version, and am still getting the same error: this from the livy logs:
17/03/21 17:35:02 ERROR RSCClient: Failed to connect to context. java.util.concurrent.TimeoutException: Timed out waiting for context to start. at com.cloudera.livy.rsc.ContextLauncher.connectTimeout(ContextLauncher.java:126) at com.cloudera.livy.rsc.ContextLauncher.access$100(ContextLauncher.java:62) at com.cloudera.livy.rsc.ContextLauncher$1.run(ContextLauncher.java:114) at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:123) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:380) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) at java.lang.Thread.run(Thread.java:745) 17/03/21 17:35:02 INFO RSCClient: Failing pending job 22872eb6-951b-43aa-8df0-21dc9b4d5274 due to shutdown. 17/03/21 17:35:17 INFO RSCAppListener: Trying to stop app application_1490063492900_0005 17/03/21 17:35:17 ERROR RSCClient: Failed to connect to context.