hi   were you able to resolve this problem?   ip-10-0-0-5.ec2.internal, executor 1): java.lang.AbstractMethodError at org.apache.spark.Logging$class.log(Logging.scala:50) at org.apache.spark.streaming.twitter.TwitterReceiver.log(TwitterInputDStream.scala:60) at org.apache.spark.Logging$class.logInfo(Logging.scala:58) at org.apache.spark.streaming.twitter.TwitterReceiver.logInfo(TwitterInputDStream.scala:60) at org.apache.spark.streaming.twitter.TwitterReceiver.onStart(TwitterInputDStream.scala:96) at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.s  ... View more

I was able to resolve two days back and found that CDH is using old twitter4j jars. Below jars are present at /usr/lib/flume-ng/lib[qucik start vm] which are causing run time conflicts.    twitter4j-core-3.0.3.jar twitter4j-stream-3.0.3.jar ... View more

I used these options and the original expection is resolved but now getting below error. Any way to resolve it?   spark.executor.userClassPathFirst true spark.driver.userClassPathFirst true   g parents 16/11/07 10:59:45 INFO storage.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 2.8 KB, free 2.8 KB) Exception in thread "dag-scheduler-event-loop" java.lang.UnsatisfiedLinkError: org.xerial.snappy.SnappyNative.maxCompressedLength(I)I at org.xerial.snappy.SnappyNative.maxCompressedLength(Native Method) at org.xerial.snappy.Snappy.maxCompressedLength(Snappy.java:316) at org.xeria ... View more

I have to use twitter4j 4.0.4.  I tried now like this but still got the error. conf.setJars(Array("/home/cloudera/Documents/lib/twitter/twitter4j-core-4.0.4.jar","/home/cloudera/Documents/lib/twitter/twitter4j-stream-4.0.4.jar"))  twitter4j 3.0.3 is for some reason creating conflict which I am not able to resolve. I do not need 3.0.3. ... View more

Thanks for your reply.   I have put these entries in build.sbt.  libraryDependencies += "org.twitter4j" % "twitter4j-stream" % "4.0.4" libraryDependencies += "org.twitter4j" % "twitter4j-core" % "4.0.4"  There is no issue when I run it in Intellij but when I package it and run it in CDH5.8 VM i get the error.  ... View more

I am also having the same problem on quickstart CDH5.8.  Even if I use --jars with the spark-submit I get the same problem.    These jars I am using- spark-streaming-twitter_2.10-1.6.0-cdh5.8.0.jar twitter4j-core-4.0.4.jar twitter4j-stream-4.0.4.jar   error ver.ReceiverSupervisorImpl: Starting receiver 16/11/06 19:07:15 ERROR executor.Executor: Exception in task 0.0 in stage 0.0 (TID 0) java.lang.NoSuchMethodError: twitter4j.TwitterStream.addListener(Ltwitter4j/StreamListener;)V at org.apache.spark.streaming.twitter.TwitterReceiver.onStart(TwitterInputDStream.scala:72) at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:148) at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:130) at org.apache.spark.st \ ... View more