Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to keep kafka producer running continuously ?

avatar
Master Guru

If I use below command to get data from my_file.txt, it captures all the data and I can get it at consumer console. If my_file.txt is continuosly updating then how can I keep my producer running and keep producing latest data?

kafka-console-producer.sh --broker-list localhost:9092 --topic my_topic < my_file.txt
1 ACCEPTED SOLUTION

avatar
Super Guru

did you try tail -f file | kafka-console-producer.sh --broker-list localhost:9092--topic my_topic

View solution in original post

4 REPLIES 4

avatar
Super Guru

did you try tail -f file | kafka-console-producer.sh --broker-list localhost:9092--topic my_topic

avatar
Master Guru

@Rajkumar Singh - Thank you! It works.

avatar
Master Mentor

I would also use NiFi's TailFile processor and PutKafka to achieve what you're doing.

avatar
Master Guru

Thank you @Artem Ervits