Cannot find SparkR executable running R snippet from Hue Notebook

[#1]

I am running hadoop 2.7.2 with spark 1.6.1 on Yarn. I have built hue and livy from the source at http://github.com/cloudera/hue. I built livy against spark 1.6.1 using:

mvn -DskipTests -Dspark.version=1.6.1 clean package

I am trying to run the Hue Notebook Sample (or my simple own code). I am able to run hive queries, pySpark, scala, but not R. I get the following error in the container log on R snippets:

Exception thrown when initializing server: java.lang.Exception: Cannot find sparkR executable

SPARK_HOME and other environment variables are set. I am also able to run sparkR shell on yarn or local from the terminal.

I figured the exception was coming from hue/apps/spark/java/livy-repl/src/main/scala/com/cloudera/hue/livy/repl/sparkr/SparkRInterpreter.scala as it was not finding the executable. I replaced the "sparkR" with the complete path to sparkR to see what happens, built again, but then I get the following error in the container log, after waiting 10000ms for Spark Context to Initialize. Appreciate any hints on what might be going wrong? I do not see any other useful logs on yarn or spark.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/hadoop-root/nm-local-dir/usercache/root/filecache/18/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/mydh/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/05/07 15:59:55 INFO yarn.ApplicationMaster: Registered signal handlers for [TERM, HUP, INT]
16/05/07 15:59:56 INFO yarn.ApplicationMaster: ApplicationAttemptId: appattempt_1462660966341_0003_000001
16/05/07 15:59:57 INFO spark.SecurityManager: Changing view acls to: root
16/05/07 15:59:57 INFO spark.SecurityManager: Changing modify acls to: root
16/05/07 15:59:57 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)
16/05/07 15:59:57 INFO yarn.ApplicationMaster: Starting the user application in a separate Thread
16/05/07 15:59:57 INFO yarn.ApplicationMaster: Waiting for spark context initialization
16/05/07 15:59:57 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 
16/05/07 15:59:57 INFO util.log: Logging initialized @2184ms
16/05/07 15:59:57 INFO server.Server: jetty-9.2.z-SNAPSHOT
16/05/07 15:59:57 INFO servlet.ScalatraListener: The cycle class name from the config: com.cloudera.hue.livy.repl.ScalatraBootstrap
16/05/07 15:59:57 INFO servlet.ScalatraListener: Initializing life cycle class: ScalatraBootstrap
16/05/07 15:59:57 ERROR sparkr.SparkRInterpreter: Process has died with 127
16/05/07 15:59:57 INFO repl.ScalatraBootstrap: Notifying http://127.0.0.1:8998/sessions/1/callback that we're up
16/05/07 15:59:57 INFO handler.ContextHandler: Started c.c.h.j.s.ServletContextHandler@168d645d{/,file:/tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1462660966341_0003/container_1462660966341_0003_01_000001/src/main/com/cloudera/hue/livy/repl,AVAILABLE}
16/05/07 15:59:57 INFO server.ServerConnector: Started ServerConnector@1bd85653{HTTP/1.1}{0.0.0.0:35014}
16/05/07 15:59:57 INFO server.Server: Started @2697ms
16/05/07 15:59:57 INFO livy.WebServer: Starting server on 35014
16/05/07 16:00:07 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 
16/05/07 16:00:17 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 
16/05/07 16:00:27 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 
16/05/07 16:00:37 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 
16/05/07 16:00:47 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 
16/05/07 16:00:57 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 
16/05/07 16:01:07 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 
16/05/07 16:01:17 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 
16/05/07 16:01:27 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 
16/05/07 16:01:37 ERROR yarn.ApplicationMaster: SparkContext did not initialize after waiting for 100000 ms. Please check earlier log output for errors. Failing the application.
16/05/07 16:01:37 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 13, (reason: Timed out waiting for SparkContext.)
16/05/07 16:01:37 INFO util.ShutdownHookManager: Shutdown hook called
16/05/07 16:01:37 INFO server.ServerConnector: Stopped ServerConnector@1bd85653{HTTP/1.1}{0.0.0.0:0}
16/05/07 16:01:37 INFO servlet.ScalatraListener: Destroying life cycle class: com.cloudera.hue.livy.repl.ScalatraBootstrap
16/05/07 16:01:37 INFO handler.ContextHandler: Stopped c.c.h.j.s.ServletContextHandler@168d645d{/,file:/tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1462660966341_0003/container_1462660966341_0003_01_000001/src/main/com/cloudera/hue/livy/repl,UNAVAILABLE}

[#2]

Also, this happens in both yarn and process modes. Appreciate any help!

[#3]

Update: Going back to Spark 1.5.0 solved the second problem (I read on another post that 1.6.1 is not supported in notebook). But I still had to change the path to the complete path for sparkR.

[#4 · Accepted solution]

I made sparkR available in the path and it works. Also, had to go back to spark 1.5.0

[#5]

I'm happy to see you resolved the issue. Thanks for sharing the solution!

[#6]

Hi  BhushanK

I am having the same problem that you reported in your first email.
Please could give all details, how do you pointed the sparkR path? What files have you configured?
Because I have tried and doesnt work.

I really appreciate your answer .
Thank you very much.

Marcus

marcus@estanislao.com.br

[#7]

Hi @Estannislao

Like I noted, I had to go back to spark 1.5.0. Next, I added the complete path to spark/bin to my PATH environment variable. I did not have to change anything in code.

Regards,

Bhushan

[#8]

Hi BhushanK

Thank you very much.

Please, How do you setup the path in hue.sh e hue.ini is it like this?

hue.sh

export HUE_HOME=/opt/mapr/hue/hue-3.9.0
export HADOOP_CONF_DIR=/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop

hue.ini

[spark]
livy_server_host=localhost

# Configure livy to start with 'process', 'thread', or 'yarn' workers.
livy_server_session_kind=yarn