Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Does Nifi always create a Content Repository on Disk?

Does Nifi always create a Content Repository on Disk?

New Contributor

I thought Nifi always created a Content Repository on Disk. I realize there are settings to limit the Flowfile Repository swapping and synchronization but I thought the Content Repository was always created. Is this true or can Nifi be configured to keep even the Content Repository in memory?

3 REPLIES 3

Re: Does Nifi always create a Content Repository on Disk?

Super Guru

my previous answer was for another question. Flow files (via hashmap) are stored in memory and time to time swapped to disk based on configurable setting (nifi.queue.swap.threshold). The content is store on disk, not in memory. If you want promote content into memory, that is essentially via flow file attributes (update attribute processor). Otherwise the content will remain on disk "until it is aged off (deleted after a certain amount of time) or deleted due to the Content Repo taking up too much space."

Re: Does Nifi always create a Content Repository on Disk?

New Contributor

@smanjee

A coworker pointed out each Repository has a Volatile implementation option.

nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository (org.apache.nifi.controller.repository.VolatileFlowFileRepository) nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository (org.apache.nifi.controller.repository.VolatileContentRepository) nifi.provenance.repository.implementation=org.apache.nifi.provenance.PersistentProvenanceRepository (org.apache.nifi.provenance.VolatileProvenanceRepository)

If you use the Volatile option for each repository then Nifi becomes totally (or mostly) memory resident? You lose state/data info on reboots etc. but can Nifi run in a memory only configuration reliably otherwise is my question.

Re: Does Nifi always create a Content Repository on Disk?

Super Guru
@J.Thomas King

At the risk of losing data yu can also use Volatile Content Repository. This is not recommended for obvious risks of losing data.

https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#volatile-content-repository-pr...