How to prevent nifi-app logs from filling my drive on my cluster

gizo — Mon, 07 Nov 2022 09:46:16 GMT

Have been getting the below exception "ReplaceText[id=3fa43c98-5a5a-1c87-8e34-911a06de02aa] Processing failed: org.apache.nifi.processor.exception.ProcessException: IOException thrown from ReplaceText[id=3fa43c98-5a5a-1c87-8e34-911a06de02aa]: java.io.FileNotFoundException: /opt/nifi-1.17.0/content_repository/466/1666653401958-112082 (No space left on device) - Caused by: java.io.FileNotFoundException: /opt/nifi-1.17.0/content_repository/466/1666653401958-112082 (No space left on device)"

It seems like the disk is full which is not allowing the processors to update or modify the data. I notice that the logs directory on all the cluster have taken all the space.

Each node has 64G of storage space and log directory has taken almost 55G of space.

I recently notice my disk is getting full very quickly. I remove the logs and the repository and restart nifi and that seems to work but then again after a few hours it got filled up. I finally notice that the logs directly is taking all the space. How to prevent nifi-app logs from filling my drive.

Re: How to prevent nifi-app logs from filling my drive on my cluster

ckumar — Mon, 07 Nov 2022 12:21:46 GMT

You can refer to this article and tune the logback.xml file to delete and rollover based log files based on you space constraints https://community.cloudera.com/t5/Community-Articles/Understanding-how-the-logback-xml-configuration-in-NiFi/ta-p/248662

If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post.

Thank you,

Chandan

question How to prevent nifi-app logs from filling my drive on my cluster in Support Questions

How to prevent nifi-app logs from filling my drive on my cluster

Re: How to prevent nifi-app logs from filling my drive on my cluster