If you are interested using Confluence Kafka, there are so many plugin (Source & Sink). here is the plugin to push the data to HDFS - https://www.confluent.io/connector/kafka-connect-hdfs/ But to use these plugins it is better to use the kafa bundle from confluence site itself - https://www.confluent.io/download/ To the actual question "Loading the data from csv / excel to Kafka topic" - Here is the commands that can help you.. Note: Try Both starting Consumer before starting Producer or use the option in consumer "--from-beginning" From Producer: /usr/hdp/2.4.2.0-258/kafka/bin$ cat /tmp/a.txt Am,sending, this , information,for,testing,purpose This,is,for,helping,others /usr/hdp/2.4.2.0-258/kafka/bin$ ./kafka-console-producer.sh --topic test --broker-list c3des330:6667 < /tmp/a.txt From Consumer: /usr/hdp/2.4.2.0-258/kafka/bin$ ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic test {metadata.broker.list=c3des266:6667,c3des330:6667, request.timeout.ms=30000, client.id=console-consumer-64407, security.protocol=PLAINTEXT} Am,sending, this , information,for,testing,purpose This,is,for,helping,others Hope this helps.
... View more
