Created on 10-26-201711:40 PM - edited 08-17-201910:27 AM
Learn how to consume real-time data from the Satori RTM platform using NiFi.
Satori is a cloud-based live platform that provides a publish-subscribe messaging service called RTM, and also makes available a set of free real-time data feeds as part of their Open Data Channels initiative:
That's it! after starting the ConsumeSatoriRtm process you will see data flowing:
The processor also supports using Satori's Streamview filters, which allow you to provide SQL-like queries to select, transform, or aggregate messages from a subscribed channel:
In the 'big-rss' example above, the following filter configuration would limit the stream to messages containing the word "jobs".
select * from `big-rss` where feedURL like '%jobs%'
The NiFi processor also supports batching of multiple messages into a single FlowFile, which will provide a new-line delimited list of messages in each file (based on a 'minimum batch size' configuration):