question Re: NiFi SFTP Input missing in Support Questions

NiFi SFTP Input missing

mbraunerde — Tue, 05 Apr 2022 08:36:54 GMT

Hey guys,

i've been a quite reader in this forums last months and you all provide me a lot of hints and solutions for multiple questions.

Now i've search the documentation and this community, but i'm the first one with this problem i think.

We store all our remote-server ips in a database, the login is the same for all. We want to fetch files (names changes possible) from remote servers and download them.

The only (built in) sftp processor with input connection is "GetSFTP", but i have to know the filename to perform a download. i cannot guarantee that the filenames will never change, so this is no way to solve my problem ...

I know, i can create an processor on my own with some development, but my first step is to get it work with built-in ones.

Does anybody got a good idea how to solve it or have i to create a processor on my own?

Greetings from Dortmund

Re: NiFi SFTP Input missing

steven-matison — Tue, 05 Apr 2022 16:15:10 GMT

@mbraunerde Before GetSFTP, you need this processor first:

ListSFTP

ListSFTP will send GetSFTP the the list of files to get. Be sure to search in your NiFi Processor window, SFTP, to see all matching processors. This works for any keyword, and I use it often.

Docs here: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.16.0/org.apache.nifi.processors.standard.ListSFTP/index.html

Re: NiFi SFTP Input missing

mbraunerde — Tue, 05 Apr 2022 18:59:36 GMT

Hey @steven-matison ,

thanks for replying, but that’s my problem - getsftp provides files and listsftp the data to get it, but listsftp doesn’t allow incoming connections to provide the option per server based on a sql query.

listsftp doc says

Input requirement:

This component does not allow an incoming relationship.

and that’s the point - I need a way to provide filename to fetch or ip to connect 🤷🏼‍:male_sign:

Re: NiFi SFTP Input missing

steven-matison — Wed, 06 Apr 2022 12:39:17 GMT

Ahh that certainly is a different challenge which would require a slightly different approach. My best recommendation, outside of making your own processors, would be a combination of api calls to prepare variables for a processor group that uses listsftp and fetchsftp or getsftp once the variables are setup.

Re: NiFi SFTP Input missing

mbraunerde — Wed, 06 Apr 2022 13:14:13 GMT

Hey steven-matison,

Thank you for the reply, but I think that way is more complicated to get it supported over multiple versions and easy to understand for my next-year-me.

Re: NiFi SFTP Input missing

mbraunerde — Tue, 02 Aug 2022 14:03:13 GMT

I've created a simple processor on my own => https://github.com/mbraunerDE/nifi-plugins/blob/4cd76e047af88e2ec84f51c883b40f74d1bb9ef0/nifi-mbrauner-plugins-processors/src/main/java/de/mbrauner/nifiplugins/processors/ListSFTPWithInput.java