Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Using Spark Phoenix Connecter from Zeppelin

Highlighted

Using Spark Phoenix Connecter from Zeppelin

New Contributor

Hi.

I'm using Zeppelin with a fairly fresh install of HDP 2.6.4 sandbox, trying to get it to read from Phoenix using the Spark Phoenix connector. I've added the jars /usr/hdp/current/phoenix-server/phoenix-spark2.jar and /usr/hdp/current/phoenix-server/phoenix-client.jar to the spark2 interpreter dependencies for Zeppelin, and then created the table US_POPULATION from the Phoenix example here:

https://phoenix.apache.org/Phoenix-in-15-minutes-or-less.html

I then try to load this table in Spark Zeppelin:

val df = sqlContext.read
.format("org.apache.phoenix.spark")
.option("table", "US_POPULATION")
.option("zkUrl", "localhost:2181:/hbase-unsecure")
.load()

df.show

I get the stack trace below. I'm guessing this has to do with the order in which the classes are loaded -- I have read that phoenix-spark2.jar has to load before phoenix-client.jar, and I've put them in that order in the list of dependencies for the spark2 interpreter, but I'm unsure how that translates into the actual load order (I put them in the opposite order too, no difference). I'm wondering if anyone has a suggestion for how to get this to work.

Thanks,

tavis

df: org.apache.spark.sql.DataFrame = [STATE: string, CITY: string ... 1 more field] org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 3, sandbox-hdp.hortonworks.com, executor 1): java.lang.NoClassDefFoundError: org/apache/spark/sql/DataFrame at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) at java.lang.Class.getDeclaredMethod(Class.java:2128) at java.io.ObjectStreamClass.getPrivateMethod(ObjectStreamClass.java:1575) at java.io.ObjectStreamClass.access$1700(ObjectStreamClass.java:79) at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:508) at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:482) at java.security.AccessController.doPrivileged(Native Method) at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:482) at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:379) at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:669) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1875) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1744) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2032) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75) at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:80) at org.apache.spark.scheduler.Task.run(Task.scala:108) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:338) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.DataFrame at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 172 more Driver stacktrace: at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$failJobAndIndependentStages(DAGScheduler.scala:1517) at org.apache.spark.scheduler.DAGScheduler$anonfun$abortStage$1.apply(DAGScheduler.scala:1505) at org.apache.spark.scheduler.DAGScheduler$anonfun$abortStage$1.apply(DAGScheduler.scala:1504) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1504) at org.apache.spark.scheduler.DAGScheduler$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:814) at org.apache.spark.scheduler.DAGScheduler$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:814) at scala.Option.foreach(Option.scala:257) at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:814) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1732) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1687) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1676) at org.apache.spark.util.EventLoop$anon$1.run(EventLoop.scala:48) at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:630) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2029) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2050) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2069) at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:336) at org.apache.spark.sql.execution.CollectLimitExec.executeCollect(limit.scala:38) at org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$collectFromPlan(Dataset.scala:2861) at org.apache.spark.sql.Dataset$anonfun$head$1.apply(Dataset.scala:2150) at org.apache.spark.sql.Dataset$anonfun$head$1.apply(Dataset.scala:2150) at org.apache.spark.sql.Dataset$anonfun$55.apply(Dataset.scala:2842) at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:65) at org.apache.spark.sql.Dataset.withAction(Dataset.scala:2841) at org.apache.spark.sql.Dataset.head(Dataset.scala:2150) at org.apache.spark.sql.Dataset.take(Dataset.scala:2363) at org.apache.spark.sql.Dataset.showString(Dataset.scala:241) at org.apache.spark.sql.Dataset.show(Dataset.scala:637) at org.apache.spark.sql.Dataset.show(Dataset.scala:596) at org.apache.spark.sql.Dataset.show(Dataset.scala:605) ... 47 elided Caused by: java.lang.NoClassDefFoundError: org/apache/spark/sql/DataFrame at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) at java.lang.Class.getDeclaredMethod(Class.java:2128) at java.io.ObjectStreamClass.getPrivateMethod(ObjectStreamClass.java:1575) at java.io.ObjectStreamClass.access$1700(ObjectStreamClass.java:79) at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:508) at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:482) at java.security.AccessController.doPrivileged(Native Method) at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:482) at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:379) at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:669) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1875) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1744) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2032) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:479) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2168) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2277) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2201) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2059) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1566) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:426) at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75) at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:80) at org.apache.spark.scheduler.Task.run(Task.scala:108) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:338) ... 3 more Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.DataFrame at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 172 more

Don't have an account?
Coming from Hortonworks? Activate your account here