What is the difference between publishKafka and publishKafkaRecord processors in Apache NiFi

3 replies
4493 views
Translate All

What is the difference between using the PublishKafka processor versus the PublishKafkaRecord processor when sending data from Apache NiFi to Kafka?

aakulovAnswer

Platinum ⭐️

PublishKafka sends the entire flow file to Kafka as a single message. PublishKafkaRecord splits each "record" (e.g. row) in the flow file and sends each record as a message.

References:

1. PublishKafka: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-kafka-0-9-nar/1.6.0/org.apache.nifi.processors.kafka.pubsub.PublishKafka/

2. PublishKafkaRecord: https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-kafka-2-0-nar/1.9.0/org.apache.nifi.processors.kafka.pubsub.PublishKafkaRecord_2_0/index.html