Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NiFi Repository - Typical Disk Usage Ratios among the repositories

avatar
Expert Contributor

Do we have any experience on typical disk usage rations for each of the repositories (Flow file, content, and provenance)? E.g. if Content requires 200 GB of storage, the provenance and flow file would require 20 GB (for typical flows)?

Trying to use this information to decide how best to slice of a NiFi server which has 12 local drives. E.g. 8 drives allocated for Content, 2 for flow file, and 2 for provenance.

Appreciate any thoughts!

1 ACCEPTED SOLUTION

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
2 REPLIES 2

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Super Mentor

for you scenario with 12 disks (assuming all disk are 200 GB)

You can specify/define multiple Content repos and multiple Provenance repos; however, you can only define one FlowFile repository and one database repository.

- 8 disks for Content repos:

- /cont_repo1 <-- 200 GB

- /cont_repo2 <-- 200 GB

- /cont_repo3 <-- 200 GB

- /cont_repo4 <-- 200 GB

- /cont_repo5 <-- 200 GB

- /cont_repo6 <-- 200 GB

- /cont_repo7 <-- 200 GB

- /cont_repo8 <-- 200 GB

- 2 disks for Provenance repos:

- /prov_repo1 <-- 200 GB

- /prov_repo2 <-- 200 GB

- 1 disk split into multiple partitions for:

- /var/log/nifi-logs/ <-- 100 GB

- OS partitions <-- split amongst other Standard OS (/tmp, /, etc...)

- 1 disk split into multiple partitions for:

- /opt/nifi <-- 50 GB

- /flowfile_repo/ <-- 50 GB

- /database_repo/ <-- 25 GB

- /opt/configuration-resources <-- 25 GB (this will hold any certs, config files, extras your NiFi processors/ dataflows may need).