Hi , All
After we upgarding NiFi 1.9.2 to 1.22.0 (using cluster) , we found that ListFile will lost tracking. Just like @TimothySpann say.
see: https://community.cloudera.com/t5/Support-Questions/NiFi-upgrade-1-9-2-to-1-23-0/m-p/376353#M242873
Here is our ListFile setting :
Listing Strategy Tracking Entities
Recurse Subdirectories true
Record Writer No value set
Input Directory Location Remote
Include File Attributes true
Minimum File Age 1 min
Maximum File Age No value set
Maximum File Size No value set
Ignore Hidden Files true
Target System Timestamp Precision Auto Detect
Entity Tracking State Cache RedisDistributedMapCacheClientService
Entity Tracking Time Window 365 days
Entity Tracking Initial Listing Target All Available
Entity Tracking Node Identifier ${hostname()}
Maximum Number of Files to Track 100000
Max Disk Operation Time 10 secs
Max Directory Listing Time 3 mins
We want to know thet reason why upgrade will cause lost tracking and duplicate files.
Is that isListingResetNecessary(see below) be triggered that cause duplicate files ?
https://github.com/apache/nifi/blob/rel/nifi-1.22.0/nifi-nar-bundles/nifi-standard-bundle/nifi-stand...
There are state save in redis －＞ListedEntities::44793129-057f-1c10-af50-b96588dde6d7.
But it seems to not using this k=v in new NiFi
