Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Nifi - executeSQL process creates thousands messaages

avatar
Expert Contributor

Hi,

I ran a simple query with executeSql process (select * from piers.status) but it created 10k duplicate messages in the success queue and all the massages are the same.

Just wondering if i missed anything in the configuration

Thanks

SJ

1 ACCEPTED SOLUTION

avatar
Contributor

By default, processor runs same SQL every 0 seconds.

Please set Scheduling > Run Schedule to a higher value and make sure that SQL is created in such a way that upon execution it picks only new/updated records else you will see "already processed" data.

Also, if NiFi is clustered then please set Scheduling > Execution to Primary node, else all nodes will run same query and you will see each record being processed "n" times where "n" = No. of nodes in Cluster.

View solution in original post

1 REPLY 1

avatar
Contributor

By default, processor runs same SQL every 0 seconds.

Please set Scheduling > Run Schedule to a higher value and make sure that SQL is created in such a way that upon execution it picks only new/updated records else you will see "already processed" data.

Also, if NiFi is clustered then please set Scheduling > Execution to Primary node, else all nodes will run same query and you will see each record being processed "n" times where "n" = No. of nodes in Cluster.