Files timestamp corresponds to month. All these monthly folders continuously gets data.
WhenListFileprocessor ingest files from201911directory and new files gets added into other folders (folders older than 201911, say 201903) these files are not picked byListFileprocessor. I tried using different values for Entity Tracking Time Window property but no luck. Apparently Tracking Entities Listing Strategy is behaving like Tracking Timestamps(caching latest timestamp from ingested files and not ingesting any older timestamped files)
As far my understanding when we useListFilewith Listing Strategy as Tracking Entities, it will cache - Name, Size and Last modified timestamp for each flowfile and then keep Listing files which are not in the cache based on these attributes.
Why ListFile processor is not picking files with my current configuration
So, basically I want to continuously ingest new files based on filename (irrespective of timestamp) and skip already ingested files - is there any workaround to achieve this.