Nifi - executeSQL process creates thousands messaages

[#1]

Hi,

I ran a simple query with executeSql process (select * from piers.status) but it created 10k duplicate messages in the success queue and all the massages are the same.

Just wondering if i missed anything in the configuration

Thanks

SJ

[#2 · Accepted solution]

By default, processor runs same SQL every 0 seconds.

Please set Scheduling > Run Schedule to a higher value and make sure that SQL is created in such a way that upon execution it picks only new/updated records else you will see "already processed" data.

Also, if NiFi is clustered then please set Scheduling > Execution to Primary node, else all nodes will run same query and you will see each record being processed "n" times where "n" = No. of nodes in Cluster.