disk io operations going up with volatile repositories

manishg — Mon, 29 Jan 2024 07:42:24 GMT

I have set following:

nifi.flowfile.repository.implementation: org.apache.nifi.controller.repository.VolatileFlowFileRepository
nifi.provenance.repository.implementation: org.apache.nifi.provenance.VolatileProvenanceRepository

With this, I expect disk io operations to go down(checked with nmon). But its just opposite and disk writes have gone up multiple times.

Why so?

Re: disk io operations going up with volatile repositories

MattWho — Mon, 29 Jan 2024 13:58:30 GMT

@manishg

Same about of flowFiles per second processing after switching to the Volatile repositories?
Perhaps having FlowFile and provenance repositories in memory allows for faster processing of FlowFIles resulting in more read and writes to the content_repository which contains the actual content of each FlowFile.

If your NiFi should crash or restart you will lose everything in your volatile repositories. The FlowFile repository holds all the FlowFile metadata for the FlowFiles currently being processed through your dataflows. This means data loss in such events.

If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

question disk io operations going up with volatile repositories in Support Questions

disk io operations going up with volatile repositories

Re: disk io operations going up with volatile repositories