Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi Acknowledgement Process

Highlighted

NiFi Acknowledgement Process

New Contributor

@bbende 

 

Hi,

 

 

I am trying to insert data in gluster replicated volume,which is impacting my Processing and Performance speed in NiFi.

When I am trying to do insertion on basic linux path using PutFile processor,It is giving good speed and Performance (5000 records/sec)

But when I am trying to insert on The path which is mount point for my gluster replicated Volume using PutFile Processor.

It is taking 250 files from the queue and wait for 2-3 minutes and only then takes next 250 files after 2 minutes.(I have set batch size to 2000 in PutFie (Processor))

 

Files are stuck in queue before putSFTP Processor from long time .

As per my understanding ,Once it picks up 250 files ,It waits for 250 files to be replicated and inserted across all replicas ,And after that process for 250 files get acknowledged only then it picks up next 250 files for processing!!

Is there any way to turn off acknowledgement process that it doesn't wait for acknowledgemnt of first 250 files insertion and replication and pick up next 250 files.


I can see the acknowledgement option in PutKafka and PutMongo Processor.
Please share how this configuration can be implemented for Putfile processor inserting data in Gluster!!

2) What is better way in production to start and stop NiFi??

Through NIFi GUI or NiFi CLI Command Script??

Don't have an account?
Coming from Hortonworks? Activate your account here