Support Questions

Find answers, ask questions, and share your expertise

NIFI ListSFTP 2.1.0 only run once

avatar
New Contributor

I am using NiFi ListSFTP 2.1.0 to scan files in the root directory of an SFTP server. The Listing Strategy is set to Tracking Timestamps, and the Remote Poll Batch Size is set to 5000. There are over 10,000 files on the server, and each file has a timestamp. The processor only outputs 1000 files during its first run, but subsequent runs do not list the remaining files. However, with the same configuration, ListSFTP 1.9.1 works as expected. Why is this happening?

 

2 REPLIES 2

avatar
Master Mentor

@lexieliu 
It may be helpful if you can share the complete configuration of your ListSFTP processor including properties and scheduling tabs.

Thanks,
Matt

avatar
New Contributor

Thanks for your reply, here is my configuration: 

lexieliu_0-1736993502289.png

lexieliu_1-1736993526186.png

lexieliu_2-1736993560989.png

I set this processor to be scheduled every minute.  When first scheduled, it output a JSON file containing 5000 data entries. However, subsequent runs did not produce any output. The timestamp recorded in the processor's status is the maximum timestamp from the initial 5000 data entries, which does not exceed the current time.