Support Questions
Find answers, ask questions, and share your expertise

How to keep kafka producer running continuously ?

Solved Go to solution

How to keep kafka producer running continuously ?

Super Guru

If I use below command to get data from my_file.txt, it captures all the data and I can get it at consumer console. If my_file.txt is continuosly updating then how can I keep my producer running and keep producing latest data?

kafka-console-producer.sh --broker-list localhost:9092 --topic my_topic < my_file.txt
1 ACCEPTED SOLUTION

Accepted Solutions

Re: How to keep kafka producer running continuously ?

did you try tail -f file | kafka-console-producer.sh --broker-list localhost:9092--topic my_topic

View solution in original post

4 REPLIES 4

Re: How to keep kafka producer running continuously ?

did you try tail -f file | kafka-console-producer.sh --broker-list localhost:9092--topic my_topic

View solution in original post

Re: How to keep kafka producer running continuously ?

Super Guru

@Rajkumar Singh - Thank you! It works.

Re: How to keep kafka producer running continuously ?

Mentor

I would also use NiFi's TailFile processor and PutKafka to achieve what you're doing.

Re: How to keep kafka producer running continuously ?

Super Guru

Thank you @Artem Ervits