I want to use source as JMS and sink to Kafka using flume, what kinds of configuration need to do?

[#1]

@vijay patel

you need JMS Source

also need Kafka Sink

and a channel of your choice

[#2]

HI Ervits Thanks for quick reply...

can you please show me flume.conf file configuration here if you dont mind

[#3 · Accepted solution]

there are examples at the links I provided

a1.sources = r1
a1.channels = c1
a1.sources.r1.type = jms
a1.sources.r1.channels = c1
a1.sources.r1.initialContextFactory = org.apache.activemq.jndi.ActiveMQInitialContextFactory
a1.sources.r1.connectionFactory = GenericConnectionFactory
a1.sources.r1.providerURL = tcp://mqserver:61616
a1.sources.r1.destinationName = BUSINESS_DATA
a1.sources.r1.destinationType = QUEUE
a1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink
a1.sinks.k1.topic = mytopic
a1.sinks.k1.brokerList = localhost:9092
a1.sinks.k1.requiredAcks = 1
a1.sinks.k1.batchSize = 20
a1.sinks.k1.channel = c1 

again you also need to add a channel, there are tons of examples on the same site. @vijay patel

[#4]

Thanks you so much

[#5]

Is it any jar file or any properties need to be changed above conf file?

[#6]

@vijay patel I suggest doing a tutorial on flume, there are no other jars necessary.

[#7]

I need to send data JMS-->kafka and kafka to HDFS using flume can you advise me what is the best option and conf file looks like?

[#8]

@vijay patel you need to open a new thread as this is different requirement from the original question. This can become messy as you will have to chain Flume agents. The only way to chain the agents is to use Avro RPC. We recommend you look at HDF powered by Apache Nifi. It will be a lot simpler to do.