Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi - FlowFile Memory dependency

NiFi - FlowFile Memory dependency

Contributor

Hi

I am trying to understand NiFi data flow mechanism . I read that Nifi has flow file which holds content and metadata (flow file attribute).

So I wanted to understand if I have 1 TB of data placed on edge node and would like to pass it to Nifi processors , is it going to load everything into memory to be used by processor?

1 REPLY 1
Highlighted

Re: NiFi - FlowFile Memory dependency

Guru

The FlowFile Repository is where NiFi keeps track of the state of what it knows about a given FlowFile that is presently active in the flow. The implementation of the repository is pluggable. The default approach is a persistent Write-Ahead Log located on a specified disk partition. Apart from the fact that NiFi itself lives in a JVM, there is no memory involvement. The processor retrieves the FlowFiles and metadata from the disk.

Don't have an account?
Coming from Hortonworks? Activate your account here