Support Questions
Find answers, ask questions, and share your expertise

How do we achieve exactly-once processing in Nifi?

Solved Go to solution
Highlighted

How do we achieve exactly-once processing in Nifi?

I have a Nifi cluster consuming messages from Kafka and sending the output to a PostgresDB. How do I guarantee that each message will be consumed/processed exactly once?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How do we achieve exactly-once processing in Nifi?

Guru
2 REPLIES 2

Re: How do we achieve exactly-once processing in Nifi?

Guru
Highlighted

Re: How do we achieve exactly-once processing in Nifi?

Here is a gross simplification that might be helpful: Exactly-once usually requires that the source system and destination system can somehow be made to agree on a method to manage at-least-once delivery and data-deduplication. NiFi could be the transport layer providing at-least-once delivery between the systems, but Kafka to NiFi without those semantics or some additional approach will not satisfy those requirements alone.

Don't have an account?