Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HDF 3.4.1 NIFI 1.9 - NIFI Provenance Repository filling disc 500GB

avatar
Explorer

HDF 3.4.1  NIFI 1.9 - NIFI Provenance Repository filling disc 500GB

 

I have a requirement to retain provenance for 5 days and made necessary changes 

 

- provenance retaining hardly - 2 days and less

content_repo - 500GB   utilization 10%
Provenence_repo - 500GB Utilization 98% 
flowfile_repo - 500GB   utilization 10%

 

below are the configs 

 

DivyaKaki_0-1603382987551.png

DivyaKaki_1-1603383062012.pngDivyaKaki_2-1603383094236.png



@MattWho @TimothySpann   Please advice 

1 ACCEPTED SOLUTION

avatar
Master Guru
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
6 REPLIES 6

avatar
Master Guru

you need to restart your servers after making the setting changes.   do you have space?

avatar
Explorer

@TimothySpann thanks for the update. restarted the cluster 2 weeks back after making changes. 
but still, the Provenance repo is piling up. surprised to see that huge disc being filled by provenance 

avatar
Master Guru

provenance is for every step of every flowfile in every flow.

 

I recommend rolling those logs

 

avatar
Explorer

Hi Tim,

Are you recommending to role nifi provenance? could you provide more pointers? 

avatar
Master Guru
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Explorer

Thanks, Tim, my whole Idea is, developers should be able to replay the message from the provenance for at least 5 days as per the requirements

I'm assuming the only solution is BUMP up the provenance storage to achieve replay capability.
please let me know your thoughts!.