Flume agent failed because dependencies were not found in classpath.

vikasreddy_a — Mon, 19 Aug 2019 10:45:47 GMT

org.apache.flume.sink.DefaultSinkFactory.create:42) - Creating instance of sink: hdfs-sink, type: hdfs 09 Mar 2016 02:07:33,594

ERROR

[conf-file-poller-0] (org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run:145)  (org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run:145)  - Failed to start agent because dependencies were not found in classpath. Error follows.
java.lang.NoClassDefFoundError: org/apache/hadoop/io/SequenceFile$CompressionType
        at org.apache.flume.sink.hdfs.HDFSEventSink.configure(HDFSEventSink.java:239)
        at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
        at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:413)
        at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:98)
        at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)

Caused by:

java.lang.ClassNotFoundException: org.apache.hadoop.io.SequenceFile$CompressionType
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

Re: Flume agent failed because dependencies were not found in classpath.

vikasreddy_a — Thu, 10 Mar 2016 00:30:17 GMT

FLUME_CLASSPATH=/root/flume/lib/

copied comon jar files from hadoop folder to the flume folder.

cp /root/hadoop/share/hadoop/common/*.jar /root/flume/lib

cp /root/hadoop/share/hadoop/common/lib/*.jar /root/flume/lib

Now the above error is rectified.

Re: Flume agent failed because dependencies were not found in classpath.

vikasreddy_a — Fri, 11 Mar 2016 19:53:59 GMT

Copied the below path in the flume-env.sh

FLUME_CLASSPATH="/home/hadoop/hadoop/share/hadoop/hdfs/"

Hdfs sink is working fine.

Re: Flume agent failed because dependencies were not found in classpath.

jack0188 — Tue, 07 Jun 2016 12:21:35 GMT

This comment fixed my problem..You are Awsome vikas

question Flume agent failed because dependencies were not found in classpath. in Archives of Support Questions (Read Only)

Flume agent failed because dependencies were not found in classpath.

Re: Flume agent failed because dependencies were not found in classpath.

Re: Flume agent failed because dependencies were not found in classpath.

Re: Flume agent failed because dependencies were not found in classpath.