how can I send send JMS to HDFS using nifi??

[#1]

you need a processor to read from JMS queue and you need a processor to put data into hdfs. you may need to handle your logic a bit but that's basically the processors you need @vijay patel

[#2 · Accepted solution]

You can pull data from JMS using either queues or topics by using these processors:

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.GetJMSTopic/in...

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.GetJMSQueue/in...

Those processors presently support ActiveMQ out of the box. It is pretty easy to add other vendors as well but we're going to make it even easier in an upcoming release.

Once you pull data from JMS then you connect that processor's output to PutHDFS as explained here:

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.hadoop.PutHDFS/index.ht...

You might find these templates useful to help get you started:

https://github.com/hortonworks-gallery/nifi-templates/tree/master/templates

Thanks

Joe

[#3]

for download nifi I can do it on windows or linux? thanks for answer

[#4]

@vijay patel you can run nifi on anything that can run jvm.

[#5]

You may also want to consider that JMS messages tend to be quite small, and before pushing to HDFS it may help to merge together many JMS messages so that you don't have a huge number of files in HDFS. You could use the MergeContent Processor to accomplish this.

[#6]

Thanks mpayne

[#7]

Configure ConsumeJMS processor, configure MergeContent processor to create suitable file sizes for HDFS, the use PutHDFS processor to write to HDFS directly.