https://community.cloudera.com/t5/Support-Questions/ClassNotFoundException-org-apache-flume-instrumentation/m-p/105710#M68588 <P>Latest 2.3 release notes <A href="http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4.7/bk_HDP_RelNotes/content/errata_flume_kafka_sink.html">http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4.7/bk_HDP_RelNotes/content/errata_flume_kafka_sink.html</A></P><P>You do not need Flume 1.6 on HDP to have it working.</P> Tue, 19 Apr 2016 21:55:14 GMT aervits 2016-04-19T21:55:14Z https://community.cloudera.com/t5/Support-Questions/ClassNotFoundException-org-apache-flume-instrumentation/m-p/105707#M68585 <P>Hi all,</P><P>We are trying to write in a Kafka queue with Flume. We have HDP 2.2.4, with this Flume 1.5.2.2.2 is installed.</P><P>Below the kafka sink configuration: </P><PRE>occLogTcp.sinks.KAFKA.type = org.apache.flume.sink.kafka.KafkaSink occLogTcp.sinks.KAFKA.topic = occTest occLogTcp.sinks.KAFKA.brokerList = &lt;broker_host_1&gt;:9092,&lt;broker_host_2&gt;:9092 occLogTcp.sinks.KAFKA.requiredAcks = 1 occLogTcp.sinks.KAFKA.batchSize = 20 occLogTcp.sinks.KAFKA.channel = c1</PRE><P>Starting the flume agent throws the following error:</P><PRE>java.lang.NoClassDefFoundError: org/apache/flume/instrumentation/kafka/KafkaSinkCounter at org.apache.flume.sink.kafka.KafkaSink.configure(KafkaSink.java:218) at org.apache.flume.conf.Configurables.configure(Configurables.java:41) at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:418) at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:103) at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)Caused by: java.lang.ClassNotFoundException: org.apache.flume.instrumentation.kafka.KafkaSinkCounter at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 12 more</PRE><P>Thank you in advance, Christian</P> Tue, 19 Apr 2016 20:05:20 GMT https://community.cloudera.com/t5/Support-Questions/ClassNotFoundException-org-apache-flume-instrumentation/m-p/105707#M68585 chri 2016-04-19T20:05:20Z https://community.cloudera.com/t5/Support-Questions/ClassNotFoundException-org-apache-flume-instrumentation/m-p/105708#M68586 <P>It seems like the package org/apache/flume/instrumentation/kafka/ has been added in version 1.6. Therefore Flume 1.5.2 doesn't support the KafkaSink.</P> Tue, 19 Apr 2016 20:44:27 GMT https://community.cloudera.com/t5/Support-Questions/ClassNotFoundException-org-apache-flume-instrumentation/m-p/105708#M68586 chri 2016-04-19T20:44:27Z https://community.cloudera.com/t5/Support-Questions/ClassNotFoundException-org-apache-flume-instrumentation/m-p/105709#M68587 <P>we backported Kafka sink into 1.5.2, just make sure you are using HDP 2.3, I see you're on 2.2.4 which is quite old. Alternatively, try HDF powered by Apache nifi</P><P><A href="https://community.hortonworks.com/questions/9425/is-flume-to-kafka-sink-supported-in-hdp-232.html" target="_blank">https://community.hortonworks.com/questions/9425/is-flume-to-kafka-sink-supported-in-hdp-232.html</A></P> Tue, 19 Apr 2016 21:44:22 GMT https://community.cloudera.com/t5/Support-Questions/ClassNotFoundException-org-apache-flume-instrumentation/m-p/105709#M68587 aervits 2016-04-19T21:44:22Z https://community.cloudera.com/t5/Support-Questions/ClassNotFoundException-org-apache-flume-instrumentation/m-p/105710#M68588 <P>Latest 2.3 release notes <A href="http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4.7/bk_HDP_RelNotes/content/errata_flume_kafka_sink.html">http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4.7/bk_HDP_RelNotes/content/errata_flume_kafka_sink.html</A></P><P>You do not need Flume 1.6 on HDP to have it working.</P> Tue, 19 Apr 2016 21:55:14 GMT https://community.cloudera.com/t5/Support-Questions/ClassNotFoundException-org-apache-flume-instrumentation/m-p/105710#M68588 aervits 2016-04-19T21:55:14Z