Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

What will cause ProducerFencedException in PublishKafka processor?

Highlighted

What will cause ProducerFencedException in PublishKafka processor?

New Contributor
PublishKafka_1_0[id=...] Failed to send StandardFlowFileRecord[uuid=fcc49ff1-178a-4cdc-aee3-edc036a61c75,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1521795710412-3, container=default, section=3], offset=0, length=9313932412],offset=0,name=xxx.txt,size=9313932412] to Kafka: org.apache.kafka.common.errors.ProducerFencedException: Producer attempted an operation with an old epoch. Either there is a newer producer with the same transactionalId, or the producer's transaction has been expired by the broker.

I got above exception in NiFi when ingesting a large (9.XX GB) file into Kafka. I would like to know what is the cause of having ProducerFencedException ?

Exception from Kafka:

 ERROR [ReplicaManager broker=0] Error processing append operation on partition DEV_D_901002_all-0 (kafka.server.ReplicaManager)
org.apache.kafka.common.errors.ProducerFencedException: Producer's epoch is no longer valid. There is probably another producer with a newer epoch. 0 (request epoch), 1 (server epoch)