Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

What will cause ProducerFencedException in PublishKafka processor?


What will cause ProducerFencedException in PublishKafka processor?

PublishKafka_1_0[id=...] Failed to send StandardFlowFileRecord[uuid=fcc49ff1-178a-4cdc-aee3-edc036a61c75,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1521795710412-3, container=default, section=3], offset=0, length=9313932412],offset=0,name=xxx.txt,size=9313932412] to Kafka: org.apache.kafka.common.errors.ProducerFencedException: Producer attempted an operation with an old epoch. Either there is a newer producer with the same transactionalId, or the producer's transaction has been expired by the broker.

I got above exception in NiFi when ingesting a large (9.XX GB) file into Kafka. I would like to know what is the cause of having ProducerFencedException ?

Exception from Kafka:

 ERROR [ReplicaManager broker=0] Error processing append operation on partition DEV_D_901002_all-0 (kafka.server.ReplicaManager)
org.apache.kafka.common.errors.ProducerFencedException: Producer's epoch is no longer valid. There is probably another producer with a newer epoch. 0 (request epoch), 1 (server epoch)
Don't have an account?
Coming from Hortonworks? Activate your account here