Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Error sparklyr from RSTUDIO SERVER

Highlighted

Error sparklyr from RSTUDIO SERVER

Super Collaborator

I have this error from RSTUDIO

---- Output Log ----
WARNING: User-defined SPARK_HOME (/opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/lib/spark) overrides detected (/opt/cloudera/parcels/SPARK2-2.0.0.cloudera1-1.cdh5.7.0.p0.113931/lib/spark2).
WARNING: Running spark-class from user-defined location.
17/08/08 23:37:09 INFO sparklyr: Session (1268) is starting under 127.0.0.1 port 8880
17/08/08 23:37:09 INFO sparklyr: Session (1268) is configuring for remote connections
Exception in thread "main" java.lang.NoSuchMethodError: scala.runtime.ObjectRef.create(Ljava/lang/Object;)Lscala/runtime/ObjectRef;
	at sparklyr.Backend.portIsAvailable(backend.scala:118)
	at sparklyr.Backend.init(backend.scala:167)
	at sparklyr.Shell$.main(shell.scala:45)
	at sparklyr.Shell.main(shell.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

My code is:

library(sparklyr)
Sys.setenv(SPARK_HOME = "/opt/cloudera/parcels/SPARK2/lib/spark2")
Sys.setenv("SPARK_HOME_VERSION" = "2.0.0")
sc <- spark_connect(version='2.0.0', master = "yarn-client", 
                    app_name = "synergic-boeing",
                    config = list("sparklyr.shell.executor-cores"=4,
                                  "sparklyr.shell.driver-cores"=4,
                                  "sparklyr.shell.executor-memory"="5G",
                                  "sparklyr.shell.driver-memory"="5G"))
Don't have an account?
Coming from Hortonworks? Activate your account here