Created on 12-04-2023 09:40 PM - edited 12-04-2023 10:05 PM
Hi , All
After we upgarding NiFi 1.9.2 to 1.22.0 (using cluster) , we found that ListFile will lost tracking. Just like @TimothySpann say.
see: https://community.cloudera.com/t5/Support-Questions/NiFi-upgrade-1-9-2-to-1-23-0/m-p/376353#M242873
Here is our ListFile setting :
We want to know thet reason why upgrade will cause lost tracking and duplicate files.
Is that isListingResetNecessary(see below) be triggered that cause duplicate files ?
https://github.com/apache/nifi/blob/rel/nifi-1.22.0/nifi-nar-bundles/nifi-standard-bundle/nifi-stand...
There are state save in redis ->ListedEntities::44793129-057f-1c10-af50-b96588dde6d7.
But it seems to not using this k=v in new NiFi
Created 12-06-2023 06:56 AM
Did you try @MattWho https://community.cloudera.com/t5/Support-Questions/NiFi-upgrade-1-9-2-to-1-23-0/m-p/376353#M242873 solution. I think best course is to always start fresh after updates and restarts to ensure nothing is lost.
Created 12-11-2023 12:40 AM
Hi, @TimothySpann
Yes, our final upgrade version is 1.22.0 and there is not ghosted processors.
We want to know the basic principle of ListFile+Redis cache mapping. We found file state had some change that stored in "redis". The change is reflected in the precision of the file's modify time.
ps. We upgrade nifi on the same machine, time is still the same.
1.9.2
{'/mnt/EDA_DATA/NIFI/cloud/upgrade_test/ControlRate5/04594': {'timestamp': 1701853172000, 'size': 0}, '/mnt/EDA_DATA/NIFI/cloud/upgrade_test/ControlRate5/04593': {'timestamp': 1701853171000, 'size': 0}}
1.22.0
{'/mnt/EDA_DATA/NIFI/cloud/upgrade_test/ControlRate5/04594': {'timestamp': 1701853172307, 'size': 0}, '/mnt/EDA_DATA/NIFI/cloud/upgrade_test/ControlRate5/04593': {'timestamp': 1701853171506, 'size': 0}}