Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to use kafka and flume in ambari ?

How to use kafka and flume in ambari ?

New Contributor

I don't know how to work with kafka and flume and how to be integrated each other ?

I want to know that how to translate data with them ? how to modify configuration file?

3 REPLIES 3
Highlighted

Re: How to use kafka and flume in ambari ?

Expert Contributor

Hi,

Apache Flume is used to collect, aggregate and move data into centralized data store. Please refer the following links for more and detailed information.

Apache Kafka is a fast, scalable, durable, and fault-tolerant publish-subscribe messaging system

http://flume.apache.org/FlumeUserGuide.html#exec-source

http://hortonworks.com/apache/kafka/

Thanks,

Mahesh

Re: How to use kafka and flume in ambari ?

Re: How to use kafka and flume in ambari ?

Rising Star

Flume is an engine most significantly used for efficient collecting, aggregating, and moving large amounts of streaming data to HDFS. As suggested above Kafka-flume sink here --> https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_HDP_RelNotes/content/ch01s08s01.html will help you push data to kafka topics via flume agents, that should be defined in the kaka-flume configs in your code

Don't have an account?
Coming from Hortonworks? Activate your account here