Hi All, I'm trying to connect my Flume to ActiveMQ and then save the data in HDFS. When I start the flume service in cloudera manager, I got the following exception: 2016-04-18 17:00:39,935 ERROR org.apache.flume.node.PollingPropertiesFileConfigurationProvider: Failed to start agent because dependencies were not found in classpath. Error follows. java.lang.NoClassDefFoundError: javax/jms/JMSException at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:190) at org.apache.flume.source.DefaultSourceFactory.getClass(DefaultSourceFactory.java:67) at org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:42) at org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:322) at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:97) at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassNotFoundException: javax.jms.JMSException at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 14 more My flume conf in cloudera manager, is this correct: flume1.sources = activemq-source-1 flume1.channels = activemq-channel-1 flume1.sinks = hdfs-sink-1 flume1.sources.activemq-source-1.type = jms flume1.sources.activemq-source-1.providerURL = tcp://10.228.200.251:61616 flume1.sources.activemq-source-1.destinationName = .broker.pub.1.00137a00000275e1.01_ILLUMINANCEMEASURE_S. flume1.sources.activemq-source-1.destinationType = TOPIC flume1.sources.activemq-source-1.channels = activemq-channel-1 flume1.sinks.hdfs-sink-1.channel = activemq-channel-1 flume1.sinks.hdfs-sink-1.type = hdfs flume1.sinks.hdfs-sink-1.hdfs.writeFormat = Text flume1.sinks.hdfs-sink-1.hdfs.fileType = DataStream flume1.sinks.hdfs-sink-1.hdfs.filePrefix = 00137a00000275e1 flume1.sinks.hdfs-sink-1.hdfs.useLocalTimeStamp = true flume1.sinks.hdfs-sink-1.hdfs.path = /user/activemq/%{topic} flume1.sinks.hdfs-sink-1.hdfs.rollCount=100 flume1.sinks.hdfs-sink-1.hdfs.rollSize=0 # specify the capacity of the memory channel. flume1.channels.activemq-channel-1.type = memory flume1.channels.activemq-channel-1.capacity = 10000 flume1.channels.activemq-channel-1.transactionCapacity = 10000 May I know how can I resolve the NoClassDefFound Exception? Do I still need to add FLUME_CLASSPATH if I added/started service in Cloudera manager? Is there any third party jar that I need to download to fix the javax/jms/JMSException? Any help is highly appreciated. Thank You.
... View more
