Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Apache nifi : ListSFTP processor raise SSH authentication error but I can list while validating

avatar
Explorer
Hello, I have an error I don't understand with ListSFTP processor. When I click on "Verify properties" for this processor it lists well an SFTP folder.
Successfully listed contents of Remote Directory [/data/folder_1/] on [hosname:port]. Found 2 objects. Of those, 2 match the filter.
But when I execute this processor I have the error I never seen:
 
ListSFTP[id=X] Processing failed: org.apache.nifi.processors.standard.socket.ClientConnectException: SSH Client connection failed [hosname:port] - Caused by: java.net.SocketTimeoutException: connect timed out
 
Why can I list when verify properties but processor doesn't work ? (Nifi version : 1.17.0)
 
I changed nothing on Nifi and don't know what to do to correct the problem or what to do to investigate.
5 REPLIES 5

avatar
Super Mentor

@lben 

if you try to ssh to the target host from the NiFi host as the NiFi service user (User that owns the running NiFi), does it connect successfully or does it timeout at command line as well?

If you made multiple connection attempts in a row from command line, do they all succeed?

Is that the complete error from the nifi-app.log?  Was it followed with a stack trace that you can share?

Thanks,

Matt

avatar
Explorer

Hello thank you for your advices. In nifi-app.log I only saw : "Stopping processor. Stopping scheduling". I never put ssh information in listsftp and it worked well before.Maybe the problem is on SFTP side. When I ssh on this SFTP server on my shell laptop it says "sftp only...".

avatar
Super Mentor

@lben 
if you saw a bulletin on the processor reporting a failure in execution, that should also be in the nifi-app.log.
You can also modify the logback.xml to change the log level of NiFi or even just the ListSFTP processor class to hopefully capture more detail on the failure.

Does SFTP to target server work from command line as the NiFi service user?  SFTP is just FTP over SSH.  But yes, SFTP servers can be configured to only allow SFTP connections.

So to get more logging out of the listSFTP processor class you could add these loggers the area where all the other loggers start to show up in the NiFi logback.xml:

 

<logger name="org.apache.nifi.processors.standard.ListSFTP" level="DEBUG"/>

 

<logger name="net.schmizz.sshj" level="DEBUG"/>
<logger name="com.hierynomus.sshj" level="DEBUG" />


Thanks,

Matt

avatar
Explorer

I work in Kubernetes. I deployed Nifi and SFTP server in other environment and everything works. So in the other environment there is a problem I don't understand. I applied the same deployment for both environment. Can the IP be blacklisted ?

avatar
Super Mentor

Not ruling out something environmental here, but what is being observed is validation working and processor execution not while both those processes should be using the same basic code.
The 3 loggers that would produce Debug logging output suggested in my previous post may shed more light on the difference in the output logging when validation is done versus running (starting) the processor.  So that is probably the best place to start.