Skip to main content
mbraunerde
mbraunerde
Gold ⭐️
April 5, 2022
Solved

NiFi SFTP Input missing

Hey guys,

 

i've been a quite reader in this forums last months and you all provide me a lot of hints and solutions for multiple questions.

 

Now i've search the documentation and this community, but i'm the first one with this problem i think.

 

We store all our remote-server ips in a database, the login is the same for all. We want to fetch files (names changes possible) from remote servers and download them.

The only (built in) sftp processor with input connection is "GetSFTP", but i have to know the filename to perform a download. i cannot guarantee that the filenames will never change, so this is no way to solve my problem ...

 

I know, i can create an processor on my own with some development, but my first step is to get it work with built-in ones.

Does anybody got a good idea how to solve it or have i to create a processor on my own?

 

Greetings from Dortmund

 

 

5 replies

steven-matison
Platinum ⭐️⭐️
steven-matison
Platinum ⭐️⭐️
April 5, 2022

@mbraunerde Before GetSFTP, you need this processor first:

 

ListSFTP 

 

ListSFTP will send GetSFTP the the list of files to get.  Be sure to search in your NiFi Processor window, SFTP, to see all matching processors.  This works for any keyword, and I use it often.

 

Docs here:  https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.16.0/org.apache.nifi.processors.standard.ListSFTP/index.html

 

 

Translate
    mbraunerde
    mbraunerdeAuthor
    Gold ⭐️
    April 5, 2022

    Hey @steven-matison ,

     

    thanks for replying, but that’s my problem - getsftp provides files and listsftp the data to get it, but listsftp doesn’t allow incoming connections to provide the option per server based on a sql query.

     

    listsftp doc says

     

    Input requirement:

    This component does not allow an incoming relationship.

     

    and that’s the point - I need a way to provide filename to fetch or ip to connect 🤷🏼‍:male_sign:

    Translate
    steven-matison
    Platinum ⭐️⭐️
    steven-matison
    Platinum ⭐️⭐️
    April 6, 2022

    Ahh that certainly is a different challenge which would require a slightly different approach.  My best recommendation, outside of making your own processors,  would be a combination of api calls to prepare variables for a processor group that uses listsftp and fetchsftp or getsftp once the variables are setup.   

    Translate
    mbraunerde
    mbraunerdeAuthor
    Gold ⭐️
    April 6, 2022
    Hey steven-matison,

    Thank you for the reply, but I think that way is more complicated to get it supported over multiple versions and easy to understand for my next-year-me.
    Translate
    Terms & ConditionsAccessibility statement