question Re: ListSFTP+FetchFTP vs GETSFTP ?? in Archives of Support Questions (Read Only)

ListSFTP+FetchFTP vs GETSFTP ??

yogesh_sharma05 — Tue, 21 Apr 2026 12:23:45 GMT

Getting files from FTP, where we can use ListSFTP and then FetchSTP to get file instead of using GetSFTP processor to get. What could be the advantage of having ListSFTP+FetchFTP over GETSFTP?

Re: ListSFTP+FetchFTP vs GETSFTP ??

Wynner — Wed, 19 Apr 2017 20:30:26 GMT

@Yogesh Sharma

FetchSFTP is cluster friendly where as GetSFTP is not, meaning you can have multiple nodes in a cluster pulling data from the same source, but with GetSFTP, you cannot.

If you have a NiFi cluster and you are using the GetSFTP processor, you would have to configure that processor to run on the primary node only so the other nodes in the cluster wouldn't try to pull the same files. This will result in the files only being pulled by a single node in a cluster instead of being pulled by all nodes, which is what FetchSFTP would do.

Do you plan to leave the files on the remote system or pull them and remove them?

Do you have a cluster or standalone instance of NiFi?

Re: ListSFTP+FetchFTP vs GETSFTP ??

yogesh_sharma05 — Wed, 19 Apr 2017 20:39:29 GMT

@Wynner

We want to keep the file there only. I am using NiFi in cluster.

Re: ListSFTP+FetchFTP vs GETSFTP ??

Wynner — Thu, 20 Apr 2017 00:44:55 GMT

@Yogesh Sharma

So, using ListSFTP retains state and will only add new files to the list when it is run, so the FetchSFTP will only pull each file once, where with GetSFTP, you would pull the same files over and over again.

Re: ListSFTP+FetchFTP vs GETSFTP ??

yogesh_sharma05 — Thu, 20 Apr 2017 13:53:33 GMT

@Wynner Thank you for the answer.

And also need one more help. Do you have any documents or reference for Best practice used in NiFi data flow development ?

Re: ListSFTP+FetchFTP vs GETSFTP ??

Wynner — Thu, 20 Apr 2017 20:07:51 GMT

@Yogesh Sharma

Here are a couple of links to articles that are a good starting point for best practices and dataflow design.

Dataflow Optimization

NiFi Best Practices for high performance

Re: ListSFTP+FetchFTP vs GETSFTP ??

pvillard — Thu, 20 Apr 2017 20:22:06 GMT

In addition to excellent resources/information provided by @Wynner, I just wanted to add:

https://pierrevillard.com/2017/02/23/listfetch-pattern-and-remote-process-group-in-apache-nifi