Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created on
06-01-2021
11:56 AM
- last edited on
06-02-2026
05:02 AM
by
VidyaSargur
What is the difference between using the PublishKafka processor versus the PublishKafkaRecord processor when sending data from Apache NiFi to Kafka?
Created 06-01-2021 12:31 PM
PublishKafka sends the entire flow file to Kafka as a single message. PublishKafkaRecord splits each "record" (e.g. row) in the flow file and sends each record as a message.
References:
1. PublishKafka: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-kafka-0-9-nar/1.6.0/org.apach...
2. PublishKafkaRecord: https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-kafka-2-...
Created 06-01-2021 12:31 PM
PublishKafka sends the entire flow file to Kafka as a single message. PublishKafkaRecord splits each "record" (e.g. row) in the flow file and sends each record as a message.
References:
1. PublishKafka: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-kafka-0-9-nar/1.6.0/org.apach...
2. PublishKafkaRecord: https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-kafka-2-...
Created 06-01-2021 12:44 PM
So if I have used JSON transformations on my data and each time 100 flow file are generated so I guess both will be equivalent ...
Created 06-01-2021 12:56 PM
If your files contain one record each, then yes, these NiFi processors will be equivalent.