Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Testing out Apache Kafka

Testing out Apache Kafka


I am currently testing on Apache Kafka, followed an example I found and managed to pump some data into Kafka.

My question now is how can I retrieve the messages in kafka and store it in hdfs?


Any advise/tutorials/examples that can share with me?




Re: Testing out Apache Kafka

Super Collaborator
Here is a blog post that gives a good walkthrough of using flume to pull messages from kafka and deliver to hdfs:

And here is the cloudera documentation detailing the options for configuring flume to work with kafka:
Don't have an account?
Coming from Hortonworks? Activate your account here