Created 07-01-2022 05:59 AM
hi
I am using custom Flume source and sink. I have created jar from that code and put it at
/$FLUME_HOME/plugin.d and define following at flume-env.sh file
FLUME_CLASSPATH="/opt/cdpdeployment/apache-flume-1.9.0-bin/plugin.d/*.jar"
but when I run below command
flume-ng agent -n agent -c -conf /opt/cdpdeployment/apache-flume-1.9.0-bin/conf/ -f /opt/cdpdeployment/apache-flume-1.9.0-bin/conf/flume-cdp.conf -Dflume.root.logger=DEBUG,console
I am getting below error:
22/07/01 16:54:15 ERROR node.PollingPropertiesFileConfigurationProvider: Failed to load configuration data. Exception follows.
org.apache.flume.FlumeException: Unable to load source type: com.cisco.cim.flume.source.CimKafkaSource, class: com.cisco.cim.flume.source.CimKafkaSource
at org.apache.flume.source.DefaultSourceFactory.getClass(DefaultSourceFactory.java:68)
at org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:42)
at org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:322)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:105)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:145)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: com.cisco.cim.flume.source.CimKafkaSource
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.flume.source.DefaultSourceFactory.getClass(DefaultSourceFactory.java:66)
... 11 more
Could you help me where I am making mistake.
I am using standalone Flume not part of Cloudera environment.
Created 07-04-2022 10:33 AM
@Uday_Singh2022 Hi, Flume has been replaced with Cloudera Flow Management (CFM). CFM is a no-code data ingestion and management solution powered by Apache NiFi. Contact your Cloudera account team for more information about moving from Flume to CFM. You can find more information HERE
Flume is not a supported component in CDP. You can find documentations on Flume on it's official website:
Regards,
Diana Torres,Created 07-07-2022 12:38 PM
@Uday_Singh2022 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks
Regards,
Diana Torres,