Member since
03-03-2020
3
Posts
0
Kudos Received
0
Solutions
03-04-2020
10:53 AM
Appreciate the feedback. Just a little more context if it helps... It's not that the ListSFTP fails but if the FetchSFTP fails to fetch what the ListSFTP provides, we aren't able to inform the ListSFTP that the file should be listed again. This may occurred due to a network outage during the transfer. The problem with the retry approach for FetchSFTP is that if the file is actually removed from the slave, we don't want to try it again. What we're trying to accomplish is something similar to rsync for multiple slaves feeding into a master server. The files need to remain on the slaves and the master should always reflect what is on the slaves. If FetchSFTP fails, then we would have outdated files on the master server.
... View more
03-04-2020
08:08 AM
Could we use an ExecuteProcessor to access the statemanager which can remove the state for a particular file that has failed. So upon FetchSFTP failure, send a flow file to an ExecuteProcessor to reach into the state and remove the entry for the file?
... View more
03-03-2020
02:24 PM
I am looking for a feedback mechanism to inform the ListSFTP processor that a transfer has failed so that it attempts to list the file again. This is an attempt for auto resolution when network issues prevent successful transfers. I was hoping to be able to use a combination of fetch/put distributed map cache and update attribute to clear the file.lastModifiedTimestamp attribute for failed flowfiles from the FetchSFTP but it seems that the distributed cache is meant only for migration of old NiFi releases.
Is there any flow that would accomplish what we are looking for?
... View more
Labels:
- Labels:
-
Apache NiFi