Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hi i want to get the streaming to kafka brokers and then send the streaming data to 2 consumers first to hdfs(raw) through flume and second to hive

Hi i want to get the streaming to kafka brokers and then send the streaming data to 2 consumers first to hdfs(raw) through flume and second to hive

 
1 REPLY 1
Highlighted

Re: Hi i want to get the streaming to kafka brokers and then send the streaming data to 2 consumers first to hdfs(raw) through flume and second to hive

Expert Contributor
@khushi kalra, you can use this https://flume.apache.org/FlumeUserGuide.html and look at the topic KafkaSource to get the data from Kafka as a source. Then use the HDFS Sink or Hive Sink to move the data into HDFS as a raw data or directly to Hive.

Alternatively you can use Kafka to store the data into HDFS. Refer to the blog: http://hortonworks.com/hadoop-tutorial/simulating-transporting-realtime-events-stream-apache-kafka/

Don't have an account?
Coming from Hortonworks? Activate your account here