Magellan Failure in Zeppelin

TimothySpann — Wed, 27 Jul 2016 23:08:58 GMT

I am running the Magellan: Geospatial Analytics on Spark zeppelin notebook and when I get to Magellan Context I get this exception.

val magellanContext = new MagellanContext(sc)

java.lang.NoClassDefFoundError: org/apache/spark/sql/sources/DataSourceStrategy$
	at org.apache.spark.sql.magellan.MagellanContext$anon$1.<init>(MagellanContext.scala:35)
	at org.apache.spark.sql.magellan.MagellanContext.<init>(MagellanContext.scala:32)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:59)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:64)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:66)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:68)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:70)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:72)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:74)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:76)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:78)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:80)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:82)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:84)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:86)
	at $iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:88)
	at $iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:90)
	at $iwC$iwC$iwC$iwC$iwC.<init>(<console>:92)
	at $iwC$iwC$iwC$iwC.<init>(<console>:94)
	at $iwC$iwC$iwC.<init>(<console>:96)
	at $iwC$iwC.<init>(<console>:98)
	at $iwC.<init>(<console>:100)
	at <init>(<console>:102)
	at .<init>(<console>:106)
	at .<clinit>(<console>)
	at .<init>(<console>:7)
	at .<clinit>(<console>)
	at $print(<console>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
	at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1346)
	at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
	at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
	at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
	at org.apache.zeppelin.spark.SparkInterpreter.interpretInput(SparkInterpreter.java:709)
	at org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:673)
	at org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:666)
	at org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:295)
	at org.apache.zeppelin.scheduler.Job.run(Job.java:171)
	at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
ERROR   
Took 2 seconds (outdated)

All the other actions before that including import and z dep work fine.

Re: Magellan Failure in Zeppelin

TimothySpann — Wed, 27 Jul 2016 23:10:31 GMT

7 months ago, it only supported 1.4 and we are running 1.6.

http://hortonworks.com/blog/magellan-geospatial-analytics-in-spark/

Github says 1.4+, not 1.4 only.

https://github.com/harsha2010/magellan

Re: Magellan Failure in Zeppelin

abajwa — Thu, 28 Jul 2016 01:47:55 GMT

@vshukla can confirm but I believe that Magellan has not been ported to Spark 1.6 yet (only 1.4)

Re: Magellan Failure in Zeppelin

krirohini — Mon, 05 Dec 2016 06:20:07 GMT

I am facing same issue with spark version 1.6.2 on hortonworks sand box, Please confirm Magellan is supported on spark 1.6 .

Thanks

Rohini

java.lang.NoClassDefFoundError: org/apache/spark/sql/sources/DataSourceStrategy$ at org.apache.spark.sql.magellan.MagellanContext$anon$1.<init>(MagellanContext.scala:35) at org.apache.spark.sql.magellan.MagellanContext.<init>(MagellanContext.scala:32) at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:39) at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:44) at $iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC$iwC.<init>(<console>:46)

I am facing same issue with spark version 1.6.2 on hortonworks sand box, Please confirm Magellan is supported on spark 1.6 .

Thanks

Rohini

Re: Magellan Failure in Zeppelin

mak88 — Fri, 08 Sep 2017 03:05:50 GMT

I am running spark version 1.6.3 under HDP 2.5.6. What version of magellan should I use to run with this version?

question Magellan Failure in Zeppelin in Archives of Support Questions (Read Only)

Magellan Failure in Zeppelin

Re: Magellan Failure in Zeppelin

Re: Magellan Failure in Zeppelin

Re: Magellan Failure in Zeppelin

Re: Magellan Failure in Zeppelin