Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError:

avatar
Expert Contributor
<code>I have installed flume.Running`flume-ng --help` shows that it's     installed. After configuring conf file for streaming data from twitter and     setting my java path in `flume-env.sh` I'm running the command to stream the data:/bin/flume-ng agent -n TwitterAgent-c conf -f /usr/local/flume/bin/conf    /flume.conf

It shows some warnings and some info from log4j. Can any one please help me in this?

<code>Logs:
 vaibhav@vaibhav-Lenovo-G570:/usr/local/flume/bin$ flume-ng agent -n          TwitterAgent-c conf -f /usr/local/flume/conf/flume.conf
+ exec /usr/local/java/bin/java -Xmx20m-cp 'conf:/usr/local/flume/lib/*'-Djava.library.path= org.apache.flume.node.Application-n     TwitterAgent-f /usr/local/flume/conf/flume.conf

log4j:WARN No appenders could be found for logger     (org.apache.flume.lifecycle.LifecycleSupervisor).

log4j:WARN Please initialize the log4j system properly.

log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
1 ACCEPTED SOLUTION

avatar

@Vaibhav Kumar

Your command is as following, Can you please explain what do you mean by "-f /usr/local/flume" in your command?

$ bin/flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf

Please refer to https://flume.apache.org/FlumeUserGuide.html and look at the argument "-f" to understand what it expects (example: conf/flume-conf.properties.template) :

 $ bin/flume-ng agent -n $agent_name -c conf -f conf/flume-conf.properties.template

View solution in original post

16 REPLIES 16

avatar
Master Guru
@Vaibhav Kumar

these log4j related warnings are okay to ignore. did you get any other error on console or in logs? if yes then can you please let us know?

avatar
Expert Contributor

@Kuldeep Kulkarni thanks for the concern .

Now i get some different error related to java path .i have give the path to jre correctly but it show some . PFA screenshot in the below comment .

vaibhav@vaibhav-Lenovo-G570:/usr/local/flume/bin$ flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume/conf/flume.conf + exec /usr/local/java/jre -Xmx20m -cp 'conf:/usr/local/flume/lib/*' -Djava.library.path= org.apache.flume.node.Application -n TwitterAgent -f /usr/local/flume/conf/flume.conf /usr/local/flume/bin/flume-ng: line 241: /usr/local/java/jre: Is a directory /usr/local/flume/bin/flume-ng: line 241: exec: /usr/local/java/jre: cannot execute: Is a directory

avatar
Expert Contributor

avatar
Master Guru

@Vaibhav Kumar

+1 to answer given by @Joy - Can you please check his answer and try to set JAVA_HOME properly ?

avatar

@Vaibhav Kumar

Looks like your JAVA_HOME is not set correctly. Thats why you are getting the following error:

/usr/local/flume/bin/flume-ng: line 241: exec: /usr/local/java/jre: cannot execute: Is a directory

From your previously added comment i think that in your case the "java" executable is present in the following path: "/usr/local/java/bin/java" Can you try using this java path?

avatar
Expert Contributor

@Joy

i have edited the path i'm getting warning from log4j if i consider this is working then why i cannot see any data in hdfs

avatar
Expert Contributor

@Joy

this is my output :

vaibhav@vaibhav-Lenovo-G570:/usr/local/flume/bin$ flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf + exec /usr/local/java/bin/java -Xmx20m -cp 'conf:/usr/local/flume/lib/*' -Djava.library.path= org.apache.flume.node.Application -n TwitterAgent -f /usr/local/flume/conf/flume.conf

log4j:WARN No appenders could be found for logger (org.apache.flume.lifecycle.LifecycleSupervisor).

log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

avatar
Expert Contributor

@Kuldeep Kulkarni i have made the changes it just showing me the logs mentioned above .No data is going to the created directory.

avatar

@Vaibhav Kumar

Looks like that your flume is not able to locate the "conf" directory properly because if it will not find the "conf/log4j.properties" then it will complain about the "log4j:WARN No appenders could be found for logger".

In your case as the "conf" directory path is not correct hence it is not able to locate the "log4j.properties" to be added to it's classpath.

Can you try starting the flume as following so that the "conf" directory relative path will be correct.

cd /usr/local/flume 

./bin/flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf

As in the above command we are in "/usr/local/flume " directory so when we pass an argument "-c conf" then it should be locate it properly. However in your case you were running the command from inside the "/usr/local/flume/bin/flume-ng -c conf" , So it was trying to find the conf directory inside "/usr/local/flume/bin/"