Skip to main content
Ytch
Ytch
Silver ⭐️⭐️
April 3, 2024
Question

using GETSFTP processor

Can someone tell me how to use GETSFTP processor?
For the hostname should we provide value like sftp://host or host
I am not able to fetch the files from SFTP.
I have provided hostname, Port, Username, Password, Remote path.

For remote path, I am following this pattern : /folder/subfolder/

    5 replies

    MattWho
    Titanium ⭐️⭐️⭐️
    MattWho
    Titanium ⭐️⭐️⭐️
    April 3, 2024

    @Ytch 

    For the "hostname" property you only provide the sftp server's fully qualified hostname.  This hostname needs to be resolvable by the NiFi host.

    What version of Apache NiFi are you using?
    What errors or exceptions do you see when you start the processor?

    Have you tried using the listSFTP processor instead?

    Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

    Thank you,
    Matt

     




    Translate
      Ytch
      YtchAuthor
      Silver ⭐️⭐️
      April 3, 2024

      When I just provide the hostname without sftp:// 
      I don't get any error as such. But I am not able to fetch the files from the provided path. But the files are present in SFTP

      Translate
        saquibsk
        saquibsk
        Gold ⭐️⭐️
        April 4, 2024

        HI @Ytch,

        • If your NiFi is clustered enviornment then make sure to run on "primary node only".

        Just cross check.

        For the hostname, you should provide the fully qualified hostname of the SFTP server. You don’t need to include the sftp:// prefix; simply provide the hostname (e.g., my-sftp-server.com). Ensure that this hostname is resolvable by the NiFi host

        •  
        Translate
        Shakib M.
          Ytch
          YtchAuthor
          Silver ⭐️⭐️
          April 4, 2024

          Hi @saquibsk 

          I am running it on primary node only.
          I tried pinging the hostname as well, it is also correct. 
          But I am not able to fetch files from the SFTP. The files exist in SFTP.


          Translate
          MattWho
          Titanium ⭐️⭐️⭐️
          MattWho
          Titanium ⭐️⭐️⭐️
          April 15, 2024

          @Ytch 

          All components on the NiFi canvas are executed as the NiFi service user and not as the user currently authenticated into the NiFi service.   So what you should do is from each host in your NiFi cluster (do on every host since any one of the hosts can be elected as the primary node at any given time), open a command prompt window/console window, become the user that owns the NiFi process, and manually ssh/sftp to the target SFTP server.  You will likely be prompted to add the target SFTP server to your known_hosts file for the NiFi service user.  NiFi SFTP processor has no way of doing this interactive step.  After successfully adding the SFTP to the known_hosts file for the NiFi service user, go back and try to start the GetSFTP or ListSFTP processors again to see if your issue is resolved.

          If not, please share your GetSFTP and ListSFTP processor component configurations.
          Also check the nifi-app.log for any exceptions or log output related to these processors.
          If no log output, you could also try enabling debug in the NiFi logback.xml for these processor classes to see what additional log output may be produced that could be useful here.

          classes for these processors are:

          org.apache.nifi.processors.standard.GetSFTP
          org.apache.nifi.processors.standard.ListSFTP

          new log lines would look like this that you would add to logback.xml:

          <logger name="org.apache.nifi.processors.standard.GetSFTP" level="DEBUG"/>
          <logger name="org.apache.nifi.processors.standard.ListSFTP" level="DEBUG"/>

          Simply add them in logback.xml where you see similar lines already.

          Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

          Thank you,
          Matt



          Translate
            Terms & ConditionsAccessibility statement