Support Questions
Find answers, ask questions, and share your expertise

Nifi - executeSQL process creates thousands messaages

Solved Go to solution
Highlighted

Nifi - executeSQL process creates thousands messaages

Rising Star

Hi,

I ran a simple query with executeSql process (select * from piers.status) but it created 10k duplicate messages in the success queue and all the massages are the same.

Just wondering if i missed anything in the configuration

Thanks

SJ

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Nifi - executeSQL process creates thousands messaages

Contributor

By default, processor runs same SQL every 0 seconds.

Please set Scheduling > Run Schedule to a higher value and make sure that SQL is created in such a way that upon execution it picks only new/updated records else you will see "already processed" data.

Also, if NiFi is clustered then please set Scheduling > Execution to Primary node, else all nodes will run same query and you will see each record being processed "n" times where "n" = No. of nodes in Cluster.

View solution in original post

1 REPLY 1

Re: Nifi - executeSQL process creates thousands messaages

Contributor

By default, processor runs same SQL every 0 seconds.

Please set Scheduling > Run Schedule to a higher value and make sure that SQL is created in such a way that upon execution it picks only new/updated records else you will see "already processed" data.

Also, if NiFi is clustered then please set Scheduling > Execution to Primary node, else all nodes will run same query and you will see each record being processed "n" times where "n" = No. of nodes in Cluster.

View solution in original post