error while fetching tweets using flume

[#1]

Hello,

I am trying to fetch tweets using flume and storing in HDFS.

following is .conf file:

TwitterAgent.sources=Twitter 
TwitterAgent.channels=MemChannel 
TwitterAgent.sinks=HDFS 
TwitterAgent.sources.Twitter.type=org.apache.flume.source.twitter.TwitterSource 
TwitterAgent.sources.Twitter.consumerKey= xxxxxxxxxxxxxxxx 
TwitterAgent.sources.Twitter.consumerSecret= xxxxxxxxxxxxxxxxxxxxxxxxxxxx 
TwitterAgent.sources.Twitter.accessToken= xxxxxxxxxxxxxxxxxxxxxx 
TwitterAgent.sources.Twitter.accessTokenSecret= xxxxxxxxxxxxxxxxxxxxxxxxx
TwitterAgent.sources.Twitter.keywords=bigdata,hadoop 
TwitterAgent.sources.Twitter.maxBatchSize=10 
TwitterAgent.sources.Twitter.maxBatchDurationMillis=10000 TwitterAgent.sources.Twitter.channels=MemChannel TwitterAgent.channels.MemChannel.type=memory 
TwitterAgent.channels.MemChannel.capacity=5000 
TwitterAgent.channels.MemChannel.transactionCapacity=10 
TwitterAgent.sinks.HDFS.channel=MemChannel 
TwitterAgent.sinks.HDFS.type=hdfs 
TwitterAgent.sinks.HDFS.hdfs.path=hdfs://sandbox.hortonworks.com:8020/Tweets/conf 
TwitterAgent.sinks.HDFS.hdfs.fileType=DataStream 
TwitterAgent.sinks.HDFS.hdfs.writeFormat=Text 
TwitterAgent.sinks.HDFS.hdfs.batchSize=10 
TwitterAgent.sinks.HDFS.hdfs.rollSize=100 
TwitterAgent.sinks.HDFS.hdfs.rollCount=10000
<br>

while executing this getting following error:

Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError: twitter4j.MediaEntity.getMediaURLHttps()Ljava/lang/String;

Twitter4J Async Dispatcher[0]) [ERROR - org.apache.flume.source.twitter.TwitterSource.onException(TwitterSource.java:331)] Exception while streaming tweets
java.lang.NullPointerException
<br>

[#2]

@heta desai

Looks like you are using Older version of "twitter4j"JARs (Specially "twitter4j-core"). So please use the latest version of this jar ... Else it will cause the same error:

Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError: twitter4j.MediaEntity.getMediaURLHttps()Ljava/lang/String;

- You will need to make sure that you are using correct version of Twitter jars.

.

- The "NullPointerException" might be a side effect of the NoSuchMethodError as the JARs that are being used does not seem to be compatible.

- Earlier version of this class had a method as following;

URL getMediaURLHttps();

Which is now changed to the following signature:

String getMediaURL();

Please see the fix as an example: https://github.com/yusuke/twitter4j/commit/86c1993a790b1c4bceda50e9675cb285c5e27b90

.