Member since
05-06-2022
5
Posts
0
Kudos Received
0
Solutions
07-01-2022
05:59 AM
hi I am using custom Flume source and sink. I have created jar from that code and put it at /$FLUME_HOME/plugin.d and define following at flume-env.sh file FLUME_CLASSPATH="/opt/cdpdeployment/apache-flume-1.9.0-bin/plugin.d/*.jar" but when I run below command flume-ng agent -n agent -c -conf /opt/cdpdeployment/apache-flume-1.9.0-bin/conf/ -f /opt/cdpdeployment/apache-flume-1.9.0-bin/conf/flume-cdp.conf -Dflume.root.logger=DEBUG,console I am getting below error: 22/07/01 16:54:15 ERROR node.PollingPropertiesFileConfigurationProvider: Failed to load configuration data. Exception follows. org.apache.flume.FlumeException: Unable to load source type: com.cisco.cim.flume.source.CimKafkaSource, class: com.cisco.cim.flume.source.CimKafkaSource at org.apache.flume.source.DefaultSourceFactory.getClass(DefaultSourceFactory.java:68) at org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:42) at org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:322) at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:105) at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:145) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: com.cisco.cim.flume.source.CimKafkaSource at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.flume.source.DefaultSourceFactory.getClass(DefaultSourceFactory.java:66) ... 11 more Could you help me where I am making mistake. I am using standalone Flume not part of Cloudera environment.
... View more
Labels:
- Labels:
-
Apache Flume
06-30-2022
01:01 AM
hi We have recently upgrade from CDH to CDP 7.1.7. Now we want to disable Kerberos on CDP cluster. Is it feasible to do so. I know it is not recommended by Cloudera to disable Kerberos. But still we want to do that to implement one use case. Could someone tell me how to disable kerberos for whole cluster in CDP 7.
... View more
Labels:
- Labels:
-
Cloudera Data Platform (CDP)
06-17-2022
05:34 AM
Thanks Guys.
... View more
06-17-2022
03:54 AM
hi @mszurap , Thanks for you reply. Yes I am aware of NiFi but client is not ready to migrate to it. They want to use Flume itself. So I am looking if there is any documentation from Cloudera side that can help.
... View more
06-16-2022
11:57 PM
Hi, I have done migration from CDH to CDP 7.1.7. We were using Apache Flume previously. But since Flume is removed from CDP, our data flow is broken. I want to know if there is any documents available for setting up standalone Flume that can connect CDP Kafka consumer and save data to HBase tables. Also want to note that I am using Kerberos for my cluster.
... View more