How to avoid record reader/writer when publishing to kafka in NiFi

[#1]

I am trying to fetch files (very small files, but millions of them) from S3 and put it into kafka and Kafka consumer will read each file from Kafka and process the data inside the file.

I noticed that publish kafka 0_11 processor forces to use record reader/writer, but for data i am trying to put into Kafka, I am not sure what will be my schema. Can I just avoid this and NiFi not try to parse this data and just pass through the flow file content and publish to kafka?

[#2 · Accepted solution]

@Praveen Bathala

Could you try considering PublishKafka_0_10 (or) PublishKafka processors which are not going to force you to specify Record Reader/Writer.

[#3]

@Shu Never noticed there is another publish processor for Kafka, PublishKafka_0_11 seems to be working fine. Thanks for the answer!