We are using Flume file channel and it keeps running out of space in few days. How do we manage the data directory size specific to a file channel. Is there some property we can use to purge old data?
I don't want to perform a manual clean-up of the file channel directories and would prefer something that will be part of the Flume Agent configuration itself.
Not sure you get me right...
Let me describe one more time:
1. We configured Flume agents(Cludera->Flume01->instance->configuration):
So our application logs looks like:
>hdfs dfs -ls /client/project/log/
And we want to keep only folders 1 month old.
So the question is how to configure clean up for sinks?