i've been a quite reader in this forums last months and you all provide me a lot of hints and solutions for multiple questions.
Now i've search the documentation and this community, but i'm the first one with this problem i think.
We store all our remote-server ips in a database, the login is the same for all. We want to fetch files (names changes possible) from remote servers and download them.
The only (built in) sftp processor with input connection is "GetSFTP", but i have to know the filename to perform a download. i cannot guarantee that the filenames will never change, so this is no way to solve my problem ...
I know, i can create an processor on my own with some development, but my first step is to get it work with built-in ones.
Does anybody got a good idea how to solve it or have i to create a processor on my own?
Greetings from Dortmund
@mbraunerde Before GetSFTP, you need this processor first:
ListSFTP will send GetSFTP the the list of files to get. Be sure to search in your NiFi Processor window, SFTP, to see all matching processors. This works for any keyword, and I use it often.
Hey @steven-matison ,
thanks for replying, but that’s my problem - getsftp provides files and listsftp the data to get it, but listsftp doesn’t allow incoming connections to provide the option per server based on a sql query.
listsftp doc says
This component does not allow an incoming relationship.
and that’s the point - I need a way to provide filename to fetch or ip to connect 🤷🏼:male_sign:
Ahh that certainly is a different challenge which would require a slightly different approach. My best recommendation, outside of making your own processors, would be a combination of api calls to prepare variables for a processor group that uses listsftp and fetchsftp or getsftp once the variables are setup.