Support Questions

Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

increase performance of putHDFS

Hello there,

I am using NiFi 1.8.0

I wanna transfer data (about 50,000,000 millions records or upper than it , 6G) from my local to HDFS.

I am using tailFile, mergContent and HDFS processors. With the default config of NiFi and processors (thread = 4) I transfer data about 2:20 ( 390 event per seconds) but data generator 's time is 900,000.

How can I increase performance for this situation?



@Matt Clarke

Please help me for this problem.

thank you


You can try to increase concurrent tasks in Processor along with heap size (in conf/bootstrap.conf). Also split your process on two. For example Tail and Merge will work on one server and PutHDFS on another. To make a link between them use port and "Remote input port" group. I hope my advice helps.

@Zholaman Kubaliyev

Thanks for your answering.

I installed NiFi on CentOS7 and I am using single NiFi instance.

heap size = 2G


for these situation, How can I solve it?

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.