How to create S3 triggers in Apache Nifi

[#1]

Hi all,

I have a scenario where I want to trigger a signal for a flow to start processing whenever there is some data available on S3 to process. In such a scenario, all processors will be EventDriven (except for the trigger), and only run if there is any data to process (or somehow we trigger them to start processing).

Scenario:

- Whenever a file(or files) lands on S3, launch SQL queries (create table, copy data etc)

So, what would be a good way for defining such a trigger?

Thanks

[#2 · Accepted solution]

Hi Obaid, please take a look at https://community.hortonworks.com/articles/49467/integrating-apache-nifi-with-aws-s3-and-sqs.html

The missing piece in your description is SQS.