Support Questions

Find answers, ask questions, and share your expertise

Flume agent failed because dependencies were not found in classpath.

avatar
Contributor

2686-flume-envsh.png

2685-classnotfoundexception.png

org.apache.flume.sink.DefaultSinkFactory.create:42) - Creating instance of sink: hdfs-sink, type: hdfs 09 Mar 2016 02:07:33,594

ERROR

[conf-file-poller-0] (org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run:145)  (org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run:145)  - Failed to start agent because dependencies were not found in classpath. Error follows.
java.lang.NoClassDefFoundError: org/apache/hadoop/io/SequenceFile$CompressionType
        at org.apache.flume.sink.hdfs.HDFSEventSink.configure(HDFSEventSink.java:239)
        at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
        at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:413)
        at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:98)
        at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744) 

Caused by:

java.lang.ClassNotFoundException: org.apache.hadoop.io.SequenceFile$CompressionType
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

hdfsconf.png
1 ACCEPTED SOLUTION

avatar
Contributor

FLUME_CLASSPATH=/root/flume/lib/

copied comon jar files from hadoop folder to the flume folder.

cp /root/hadoop/share/hadoop/common/*.jar /root/flume/lib

cp /root/hadoop/share/hadoop/common/lib/*.jar /root/flume/lib

Now the above error is rectified.

View solution in original post

3 REPLIES 3

avatar
Contributor

FLUME_CLASSPATH=/root/flume/lib/

copied comon jar files from hadoop folder to the flume folder.

cp /root/hadoop/share/hadoop/common/*.jar /root/flume/lib

cp /root/hadoop/share/hadoop/common/lib/*.jar /root/flume/lib

Now the above error is rectified.

avatar
Contributor

Copied the below path in the flume-env.sh

FLUME_CLASSPATH="/home/hadoop/hadoop/share/hadoop/hdfs/"

Hdfs sink is working fine.

avatar
Rising Star

This comment fixed my problem..You are Awsome vikas