question Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError: in Archives of Support Questions (Read Only)

Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

kumarvaibhav199 — Sun, 31 Jul 2016 03:19:10 GMT

<code>I have installed flume.Running`flume-ng --help` shows that it's     installed. After configuring conf file for streaming data from twitter and     setting my java path in `flume-env.sh` I'm running the command to stream the data:/bin/flume-ng agent -n TwitterAgent-c conf -f /usr/local/flume/bin/conf    /flume.conf

It shows some warnings and some info from log4j. Can any one please help me in this?

<code>Logs:
 vaibhav@vaibhav-Lenovo-G570:/usr/local/flume/bin$ flume-ng agent -n          TwitterAgent-c conf -f /usr/local/flume/conf/flume.conf
+ exec /usr/local/java/bin/java -Xmx20m-cp 'conf:/usr/local/flume/lib/*'-Djava.library.path= org.apache.flume.node.Application-n     TwitterAgent-f /usr/local/flume/conf/flume.conf

log4j:WARN No appenders could be found for logger     (org.apache.flume.lifecycle.LifecycleSupervisor).

log4j:WARN Please initialize the log4j system properly.

log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

KuldeepK — Sun, 31 Jul 2016 06:00:52 GMT

@Vaibhav Kumar

these log4j related warnings are okay to ignore. did you get any other error on console or in logs? if yes then can you please let us know?

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

kumarvaibhav199 — Sun, 31 Jul 2016 12:13:55 GMT

@Kuldeep Kulkarni thanks for the concern .

Now i get some different error related to java path .i have give the path to jre correctly but it show some . PFA screenshot in the below comment .

vaibhav@vaibhav-Lenovo-G570:/usr/local/flume/bin$ flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume/conf/flume.conf + exec /usr/local/java/jre -Xmx20m -cp 'conf:/usr/local/flume/lib/*' -Djava.library.path= org.apache.flume.node.Application -n TwitterAgent -f /usr/local/flume/conf/flume.conf /usr/local/flume/bin/flume-ng: line 241: /usr/local/java/jre: Is a directory /usr/local/flume/bin/flume-ng: line 241: exec: /usr/local/java/jre: cannot execute: Is a directory

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

kumarvaibhav199 — Sun, 31 Jul 2016 12:17:13 GMT

@Kuldeep Kulkarni Please find below screenshot for the above error.

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

Former Member — Sun, 31 Jul 2016 13:05:53 GMT

@Vaibhav Kumar

Looks like your JAVA_HOME is not set correctly. Thats why you are getting the following error:

/usr/local/flume/bin/flume-ng: line 241: exec: /usr/local/java/jre: cannot execute: Is a directory

From your previously added comment i think that in your case the "java" executable is present in the following path: "/usr/local/java/bin/java" Can you try using this java path?

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

KuldeepK — Sun, 31 Jul 2016 13:45:44 GMT

@Vaibhav Kumar

+1 to answer given by @Joy - Can you please check his answer and try to set JAVA_HOME properly ?

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

kumarvaibhav199 — Sun, 31 Jul 2016 14:08:03 GMT

@Joy

i have edited the path i'm getting warning from log4j if i consider this is working then why i cannot see any data in hdfs

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

kumarvaibhav199 — Sun, 31 Jul 2016 14:09:08 GMT

@Joy

this is my output :

vaibhav@vaibhav-Lenovo-G570:/usr/local/flume/bin$ flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf + exec /usr/local/java/bin/java -Xmx20m -cp 'conf:/usr/local/flume/lib/*' -Djava.library.path= org.apache.flume.node.Application -n TwitterAgent -f /usr/local/flume/conf/flume.conf

log4j:WARN No appenders could be found for logger (org.apache.flume.lifecycle.LifecycleSupervisor).

log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

kumarvaibhav199 — Sun, 31 Jul 2016 15:05:22 GMT

@Kuldeep Kulkarni i have made the changes it just showing me the logs mentioned above .No data is going to the created directory.

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

Former Member — Sun, 31 Jul 2016 16:27:55 GMT

@Vaibhav Kumar

Looks like that your flume is not able to locate the "conf" directory properly because if it will not find the "conf/log4j.properties" then it will complain about the "log4j:WARN No appenders could be found for logger".

In your case as the "conf" directory path is not correct hence it is not able to locate the "log4j.properties" to be added to it's classpath.

Can you try starting the flume as following so that the "conf" directory relative path will be correct.

cd /usr/local/flume 

./bin/flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf

As in the above command we are in "/usr/local/flume " directory so when we pass an argument "-c conf" then it should be locate it properly. However in your case you were running the command from inside the "/usr/local/flume/bin/flume-ng -c conf" , So it was trying to find the conf directory inside "/usr/local/flume/bin/"

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

kumarvaibhav199 — Sun, 31 Jul 2016 17:11:50 GMT

@Joy

I'm getting the below logs after running the above command. it is not able to instantiate the folowing class

com.webmethods.sc.logging.log4j.WmDailyFileAppender

vaibhav@vaibhav-Lenovo-G570:/usr/local/flume$ ./bin/flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf Info: Sourcing environment configuration script /usr/local/flume/conf/flume-env.sh + exec /usr/local/java/bin/java -Xms500m -Xmx1000m -Dcom.sun.management.jmxremote -cp '/usr/local/flume/conf:/usr/local/flume/lib/*:usr/local/flume/conf' -Djava.library.path= org.apache.flume.node.Application -n TwitterAgent -f /usr/local/flume /conf/flume.conf log4j:ERROR Could not instantiate class [com.webmethods.sc.logging.log4j.WmDailyFileAppender]. java.lang.ClassNotFoundException: com.webmethods.sc.logging.log4j.WmDailyFileAppender at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198) at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:326) at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:123) at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:752) at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:735) at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:615) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:502) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:547) at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483) at org.apache.log4j.LogManager.<clinit>(LogManager.java:127) at org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:73) at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:242) at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:254) at org.apache.flume.node.Application.<clinit>(Application.java:59) log4j:ERROR Could not instantiate appender named "testAppender". log4j:WARN No appenders could be found for logger (org.apache.flume.node.PollingPropertiesFileConfigurationProvider). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

Former Member — Sun, 31 Jul 2016 19:57:14 GMT

@Vaibhav Kumar

Looks like you have modified the "/usr/local/flume/conf/log4j.properties" file of your flume and added that new appender

log4j.appender.<SOME_APPENDER_NAME>=com.webmethods.sc.logging.log4j.WmDailyFileAppender

If that is the case then you will need to download the Jar that provides the above mentioned class and then that jar need to be placed inside the "/usr/local/flume/lib" directory. I do not think that the mentioned class is shipped with Flume. I even checked in latest flume "apache-flume-1.6.0-bin.tar.gz". "WmDailyFileAppender" class is not shipped with it.

Have you edited your log4j and added that appender?

I will suggest you to have a look at the log4j.properties that is shipped with flume as following and try using it:

https://github.com/apache/flume/blob/trunk/conf/log4j.properties

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

kumarvaibhav199 — Mon, 01 Aug 2016 02:15:05 GMT

@Joy Yes i had altered the log4j but i reverted it back now and cannot see any warning or errors, but no data is getting streamed to my hdfs directory.Below are the logs :

vaibhav@vaibhav-Lenovo-G570:/usr/local/flume$ bin/flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf

Info: Sourcing environment configuration script /usr/local/flume/conf/flume-env.sh + exec /usr/local/java/bin/java -Xms500m -Xmx1000m -Dcom.sun.management.jmxremote -cp '/usr/local/flume/conf:/usr/local/flume/lib/*:usr/local/flume/conf' -Djava.library.path= org.apache.flume.node.Application -n TwitterAgent -f /usr/local/flume /conf/flume.conf

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

Former Member — Mon, 01 Aug 2016 11:45:07 GMT

@Vaibhav Kumar

Your command is as following, Can you please explain what do you mean by "-f /usr/local/flume" in your command?

$ bin/flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf

Please refer to https://flume.apache.org/FlumeUserGuide.html and look at the argument "-f" to understand what it expects (example: conf/flume-conf.properties.template) :

 $ bin/flume-ng agent -n $agent_name -c conf -f conf/flume-conf.properties.template

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

kumarvaibhav199 — Mon, 01 Aug 2016 23:05:01 GMT

@Joy I edited my command ,it's as follows

bin/flume-ng agent --conf conf --conf-file flume.conf --name TwitterAgent -Dflume.root.logger=INFO,console

It worked but at the end it shows some twitter exceptions. I searched for the issue on internet it showed there is some problem with flume-sources-1.0-SNAPSHOT.jar .So i downloaded it again from a different source but still its throwing the same error.I havn't pasted the full logs here .

vaibhav@vaibhav-Lenovo-G570:/usr/local/flume$ bin/flume-ng agent --conf conf --conf-file flume.conf --name TwitterAgent -Dflume.root.logger=INFO,console Info

... Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError: twitter4j.json.JSONObjectType.determine(Ltwitter4j/internal/org/json/JSONObject;)Ltwitter4j/json/JSONObjectType; at twitter4j.AbstractStreamImplementation$1.run(AbstractStreamImplementation.java:100) at twitter4j.internal.async.ExecuteThread.run(DispatcherImpl.java:116)

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

kumarvaibhav199 — Tue, 02 Aug 2016 19:49:04 GMT

@Joy

can you please have a look.

Re: Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

KuldeepK — Wed, 03 Aug 2016 06:10:25 GMT

@Vaibhav Kumar - Can you please write a new question on this new error? I could see that reported issue got resolved. I have accepted answer by @Joy.

Please post the new question on this error and you can tag in Joy or other people in there to get quick attention 🙂