I am running hadoop 2.7.2 with spark 1.6.1 on Yarn. I have built hue and livy from the source at http://github.com/cloudera/hue. I built livy against spark 1.6.1 using:   mvn -DskipTests -Dspark.version=1.6.1 clean package   I am trying to run the Hue Notebook Sample (or my simple own code). I am able to run hive queries, pySpark, scala, but not R. I get the following error in the container log on R snippets:   Exception thrown when initializing server: java.lang.Exception: Cannot find sparkR executable SPARK_HOME and other environment variables are set. I am also able to run sparkR shell on yarn or local from the terminal.   I figured the exception was coming from hue/apps/spark/java/livy-repl/src/main/scala/com/cloudera/hue/livy/repl/sparkr/SparkRInterpreter.scala as it was not finding the executable. I replaced the "sparkR" with the complete path to sparkR to see what happens, built again, but then I get the following error in the container log, after waiting 10000ms for Spark Context to Initialize. Appreciate any hints on what might be going wrong? I do not see any other useful logs on yarn or spark.   SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/tmp/hadoop-root/nm-local-dir/usercache/root/filecache/18/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/mydh/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 16/05/07 15:59:55 INFO yarn.ApplicationMaster: Registered signal handlers for [TERM, HUP, INT] 16/05/07 15:59:56 INFO yarn.ApplicationMaster: ApplicationAttemptId: appattempt_1462660966341_0003_000001 16/05/07 15:59:57 INFO spark.SecurityManager: Changing view acls to: root 16/05/07 15:59:57 INFO spark.SecurityManager: Changing modify acls to: root 16/05/07 15:59:57 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root) 16/05/07 15:59:57 INFO yarn.ApplicationMaster: Starting the user application in a separate Thread 16/05/07 15:59:57 INFO yarn.ApplicationMaster: Waiting for spark context initialization 16/05/07 15:59:57 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 16/05/07 15:59:57 INFO util.log: Logging initialized @2184ms 16/05/07 15:59:57 INFO server.Server: jetty-9.2.z-SNAPSHOT 16/05/07 15:59:57 INFO servlet.ScalatraListener: The cycle class name from the config: com.cloudera.hue.livy.repl.ScalatraBootstrap 16/05/07 15:59:57 INFO servlet.ScalatraListener: Initializing life cycle class: ScalatraBootstrap 16/05/07 15:59:57 ERROR sparkr.SparkRInterpreter: Process has died with 127 16/05/07 15:59:57 INFO repl.ScalatraBootstrap: Notifying http://127.0.0.1:8998/sessions/1/callback that we're up 16/05/07 15:59:57 INFO handler.ContextHandler: Started c.c.h.j.s.ServletContextHandler@168d645d{/,file:/tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1462660966341_0003/container_1462660966341_0003_01_000001/src/main/com/cloudera/hue/livy/repl,AVAILABLE} 16/05/07 15:59:57 INFO server.ServerConnector: Started ServerConnector@1bd85653{HTTP/1.1}{0.0.0.0:35014} 16/05/07 15:59:57 INFO server.Server: Started @2697ms 16/05/07 15:59:57 INFO livy.WebServer: Starting server on 35014 16/05/07 16:00:07 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 16/05/07 16:00:17 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 16/05/07 16:00:27 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 16/05/07 16:00:37 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 16/05/07 16:00:47 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 16/05/07 16:00:57 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 16/05/07 16:01:07 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 16/05/07 16:01:17 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 16/05/07 16:01:27 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 16/05/07 16:01:37 ERROR yarn.ApplicationMaster: SparkContext did not initialize after waiting for 100000 ms. Please check earlier log output for errors. Failing the application. 16/05/07 16:01:37 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 13, (reason: Timed out waiting for SparkContext.) 16/05/07 16:01:37 INFO util.ShutdownHookManager: Shutdown hook called 16/05/07 16:01:37 INFO server.ServerConnector: Stopped ServerConnector@1bd85653{HTTP/1.1}{0.0.0.0:0} 16/05/07 16:01:37 INFO servlet.ScalatraListener: Destroying life cycle class: com.cloudera.hue.livy.repl.ScalatraBootstrap 16/05/07 16:01:37 INFO handler.ContextHandler: Stopped c.c.h.j.s.ServletContextHandler@168d645d{/,file:/tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1462660966341_0003/container_1462660966341_0003_01_000001/src/main/com/cloudera/hue/livy/repl,UNAVAILABLE} ... View more