Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Duplicate of flowfile after NiFi primary node re-election

avatar

I have a NiFi job starting with a GetFile processor configured to run on primary node to avoid duplicate flowfiles.

After a NiFi node restart, causing a Primary node re-election, the GetFile processor has created two flowfiles (one expected and one duplicate).

I suppose that the GetFile was configured to run with the pre-election primary node and post-election one too, causing those two flowfiles (both flowfiles were processed by different nodes according to NiFi data provenance).

Is there a way to avoid this behavior, and is it a NiFi bug ?

NB: While the job was running, coordinator and primary nodes were the same.

Thanks a lot.

1 ACCEPTED SOLUTION

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
2 REPLIES 2

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Explorer

@mattclarke

I'm having the same problem using "ScrollElasticSearchHttp" processor.

Processor state shows one or more nodes of cluster, depending on situation, even when I've configured to "Primary Node only".

Flowfiles have been duplicated on each added node.

How can I solve the problem?