Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Load terabytes of data from Local system to HDFS using Apache Nifi

Solved Go to solution

Load terabytes of data from Local system to HDFS using Apache Nifi

Rising Star

I am new in Apache NiFi. Do you have any case study to load 1 tb data from Local system to HDFS using Apache Nifi?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Load terabytes of data from Local system to HDFS using Apache Nifi

Hi @Rajib Mandal,

There is no issue at loading large volumes of data using Apache NiFi. Of course, depending of the characteristics of your setup and of your input files, the performance may change. From what you described, a simple flow with List/FetchFile processors and PutHDFS processor should perform as expected.

Hope this helps.

1 REPLY 1

Re: Load terabytes of data from Local system to HDFS using Apache Nifi

Hi @Rajib Mandal,

There is no issue at loading large volumes of data using Apache NiFi. Of course, depending of the characteristics of your setup and of your input files, the performance may change. From what you described, a simple flow with List/FetchFile processors and PutHDFS processor should perform as expected.

Hope this helps.