Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Does anyone worked on Streaming the Real Time Data from FTP Source to Kafka Producer?

Does anyone worked on Streaming the Real Time Data from FTP Source to Kafka Producer?

New Contributor

Hi All,

Does anyone worked on Streaming the Real Time Data from FTP Source to Kafka Producer? I have kafka running and trying to fetch the text data from FTP Source File. We need the Streaming data to ingest continuously in Kafka.

1 REPLY 1

Re: Does anyone worked on Streaming the Real Time Data from FTP Source to Kafka Producer?

Cloudera Employee

@Magesh Kumar Apache NiFi [1] would be a great choice for solving this problem. You can consume from an FTP source [2], process / format the data and write it into Kafka. Here is a great article on NiFi integration with Kafka [3]

[1] https://nifi.apache.org/

[2] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.GetFTP/

[3] http://bryanbende.com/development/2016/09/15/apache-nifi-and-apache-kafka