Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

[#1]

<code>I have installed flume.Running`flume-ng --help` shows that it's     installed. After configuring conf file for streaming data from twitter and     setting my java path in `flume-env.sh` I'm running the command to stream the data:/bin/flume-ng agent -n TwitterAgent-c conf -f /usr/local/flume/bin/conf    /flume.conf

It shows some warnings and some info from log4j. Can any one please help me in this?

<code>Logs:
 vaibhav@vaibhav-Lenovo-G570:/usr/local/flume/bin$ flume-ng agent -n          TwitterAgent-c conf -f /usr/local/flume/conf/flume.conf
+ exec /usr/local/java/bin/java -Xmx20m-cp 'conf:/usr/local/flume/lib/*'-Djava.library.path= org.apache.flume.node.Application-n     TwitterAgent-f /usr/local/flume/conf/flume.conf

log4j:WARN No appenders could be found for logger     (org.apache.flume.lifecycle.LifecycleSupervisor).

log4j:WARN Please initialize the log4j system properly.

log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

[#2]

@Joy

I'm getting the below logs after running the above command. it is not able to instantiate the folowing class

com.webmethods.sc.logging.log4j.WmDailyFileAppender

vaibhav@vaibhav-Lenovo-G570:/usr/local/flume$ ./bin/flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf Info: Sourcing environment configuration script /usr/local/flume/conf/flume-env.sh + exec /usr/local/java/bin/java -Xms500m -Xmx1000m -Dcom.sun.management.jmxremote -cp '/usr/local/flume/conf:/usr/local/flume/lib/*:usr/local/flume/conf' -Djava.library.path= org.apache.flume.node.Application -n TwitterAgent -f /usr/local/flume /conf/flume.conf log4j:ERROR Could not instantiate class [com.webmethods.sc.logging.log4j.WmDailyFileAppender]. java.lang.ClassNotFoundException: com.webmethods.sc.logging.log4j.WmDailyFileAppender at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198) at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:326) at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:123) at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:752) at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:735) at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:615) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:502) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:547) at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483) at org.apache.log4j.LogManager.<clinit>(LogManager.java:127) at org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:73) at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:242) at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:254) at org.apache.flume.node.Application.<clinit>(Application.java:59) log4j:ERROR Could not instantiate appender named "testAppender". log4j:WARN No appenders could be found for logger (org.apache.flume.node.PollingPropertiesFileConfigurationProvider). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

[#3]

@Vaibhav Kumar

Looks like you have modified the "/usr/local/flume/conf/log4j.properties" file of your flume and added that new appender

log4j.appender.<SOME_APPENDER_NAME>=com.webmethods.sc.logging.log4j.WmDailyFileAppender

If that is the case then you will need to download the Jar that provides the above mentioned class and then that jar need to be placed inside the "/usr/local/flume/lib" directory. I do not think that the mentioned class is shipped with Flume. I even checked in latest flume "apache-flume-1.6.0-bin.tar.gz". "WmDailyFileAppender" class is not shipped with it.

Have you edited your log4j and added that appender?

I will suggest you to have a look at the log4j.properties that is shipped with flume as following and try using it:

https://github.com/apache/flume/blob/trunk/conf/log4j.properties

[#4]

@Joy Yes i had altered the log4j but i reverted it back now and cannot see any warning or errors, but no data is getting streamed to my hdfs directory.Below are the logs :

vaibhav@vaibhav-Lenovo-G570:/usr/local/flume$ bin/flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf

Info: Sourcing environment configuration script /usr/local/flume/conf/flume-env.sh + exec /usr/local/java/bin/java -Xms500m -Xmx1000m -Dcom.sun.management.jmxremote -cp '/usr/local/flume/conf:/usr/local/flume/lib/*:usr/local/flume/conf' -Djava.library.path= org.apache.flume.node.Application -n TwitterAgent -f /usr/local/flume /conf/flume.conf

[#5 · Accepted solution]

@Vaibhav Kumar

Your command is as following, Can you please explain what do you mean by "-f /usr/local/flume" in your command?

$ bin/flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf

Please refer to https://flume.apache.org/FlumeUserGuide.html and look at the argument "-f" to understand what it expects (example: conf/flume-conf.properties.template) :

 $ bin/flume-ng agent -n $agent_name -c conf -f conf/flume-conf.properties.template

[#6]

@Joy I edited my command ,it's as follows

bin/flume-ng agent --conf conf --conf-file flume.conf --name TwitterAgent -Dflume.root.logger=INFO,console

It worked but at the end it shows some twitter exceptions. I searched for the issue on internet it showed there is some problem with flume-sources-1.0-SNAPSHOT.jar .So i downloaded it again from a different source but still its throwing the same error.I havn't pasted the full logs here .

vaibhav@vaibhav-Lenovo-G570:/usr/local/flume$ bin/flume-ng agent --conf conf --conf-file flume.conf --name TwitterAgent -Dflume.root.logger=INFO,console Info

... Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError: twitter4j.json.JSONObjectType.determine(Ltwitter4j/internal/org/json/JSONObject;)Ltwitter4j/json/JSONObjectType; at twitter4j.AbstractStreamImplementation$1.run(AbstractStreamImplementation.java:100) at twitter4j.internal.async.ExecuteThread.run(DispatcherImpl.java:116)

[#7]

@Joy

can you please have a look.

[#8]

@Vaibhav Kumar - Can you please write a new question on this new error? I could see that reported issue got resolved. I have accepted answer by @Joy.

Please post the new question on this error and you can tag in Joy or other people in there to get quick attention 🙂