Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Flume agent failed because dependencies were not found in classpath.

avatar
New Member

2686-flume-envsh.png

2685-classnotfoundexception.png

org.apache.flume.sink.DefaultSinkFactory.create:42) - Creating instance of sink: hdfs-sink, type: hdfs 09 Mar 2016 02:07:33,594

ERROR

[conf-file-poller-0] (org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run:145)  (org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run:145)  - Failed to start agent because dependencies were not found in classpath. Error follows.
java.lang.NoClassDefFoundError: org/apache/hadoop/io/SequenceFile$CompressionType
        at org.apache.flume.sink.hdfs.HDFSEventSink.configure(HDFSEventSink.java:239)
        at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
        at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:413)
        at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:98)
        at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744) 

Caused by:

java.lang.ClassNotFoundException: org.apache.hadoop.io.SequenceFile$CompressionType
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

hdfsconf.png
1 ACCEPTED SOLUTION

avatar
New Member

FLUME_CLASSPATH=/root/flume/lib/

copied comon jar files from hadoop folder to the flume folder.

cp /root/hadoop/share/hadoop/common/*.jar /root/flume/lib

cp /root/hadoop/share/hadoop/common/lib/*.jar /root/flume/lib

Now the above error is rectified.

View solution in original post

3 REPLIES 3

avatar
New Member

FLUME_CLASSPATH=/root/flume/lib/

copied comon jar files from hadoop folder to the flume folder.

cp /root/hadoop/share/hadoop/common/*.jar /root/flume/lib

cp /root/hadoop/share/hadoop/common/lib/*.jar /root/flume/lib

Now the above error is rectified.

avatar
New Member

Copied the below path in the flume-env.sh

FLUME_CLASSPATH="/home/hadoop/hadoop/share/hadoop/hdfs/"

Hdfs sink is working fine.

avatar
Rising Star

This comment fixed my problem..You are Awsome vikas