Does Falcon have reporting capabilities? Pre-built or ability to generate our own custom reports? For example, Files nearing expiration. Files that were purged last month or within 14 days etc
I've been searching for the same functionality, but I don't see it in Falcon 0.7. Looking at the source code, I see that replication statistics are saved, but not eviction statistics. I see counters being written in this file: https://github.com/apache/falcon/blob/master/replication/src/main/java/org/apache/falcon/replication..., but nothing similar in FeedEvictor.java
It would be a useful feature to send a JMS message before a file is evicted so that another process can be spawned in addition to a report of recently evicted files.
Falcon does not have reporting capabilities for listing files that are nearing expiration. If it is useful to receive an email with list of files that are purged through Retention job, Falcon has the capability to add this support. Please open a Jira.
It's not 100% what you're looking for, but we do leverage the JMS messages Falcon sends upon completion of actions like processes and replication. This allowed us to build a job history for all of our jobs. We have a client that is capturing those messages and writing them to a pseudo-database that a web app puts into a table that our team can use for supporting and monitoring jobs. It's a handy use of JMS.
You could do some combination of that with smart logic: read the retention time from an entity, compare with the current date, and send alerts for jobs letting the team know how much time is left before eviction, replication, or the next instance of anything happens.
Example attached in pic.