Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Why does my source file get read only once in the consumer console?

Why does my source file get read only once in the consumer console?

New Contributor

 

I set up everything as recommended for a quick start, I used a text file as a source or producer which contains one sentence. when I launch a consumer console for the first time I am able to read the sentence (JSON format) in the file but when I add something in the file it's not showing in the consumer console and when I use the producer console to add something in the topic, it shows right the way in the consumer console. What could be the problem?

1 REPLY 1
Highlighted

Re: Why does my source file get read only once in the consumer console?

Contributor

Hey @kaf,

 

Thanks for reaching out to the Cloudera community.

You can use "tail" command and then pipeline it to Kafka console producer if you want to read the whole file and then continue tailing for subsequently appended lines.

 

$ tail -f -n +1 <filename> | kafka-console-producer --broker-list <Broker_Host>:9092 --topic <topic_name>

 

Let me know if this helps.

Don't have an account?
Coming from Hortonworks? Activate your account here