listSFTP Failed to obtain connection to remote host due to com.jcraft.jsch.JSchException: timeout: socket is not established

[#1]

Her is my error log

2017-04-25 08:06:39,645 ERROR [Timer-Driven Process Thread-4] o.a.nifi.processors.standard.ListSFTP ListSFTP[id=fa15a338-015a-1000-ffff-ffffbc360c33] Failed to perform listing on remote host due to java.io.IOException: Failed to obtain connection to remote host due to com.jcraft.jsch.JSchException: timeout: socket is not established

2017-04-25 08:06:39,645 ERROR [Timer-Driven Process Thread-4] o.a.nifi.processors.standard.ListSFTP

java.io.IOException: Failed to obtain connection to remote host due to com.jcraft.jsch.JSchException: timeout: socket is not established

at org.apache.nifi.processors.standard.util.SFTPTransfer.getChannel(SFTPTransfer.java:447) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]

at org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:184) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]

at org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:148) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]

at org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:103) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]

at org.apache.nifi.processors.standard.AbstractListProcessor.onTrigger(AbstractListProcessor.java:341) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]

at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]

at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]

at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]

at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]

at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_77]

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_77]

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_77]

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_77]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_77]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_77]

at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]

Caused by: com.jcraft.jsch.JSchException: timeout: socket is not established

at com.jcraft.jsch.Util.createSocket(Util.java:394) ~[jsch-0.1.54.jar:na]

at com.jcraft.jsch.Session.connect(Session.java:215) ~[jsch-0.1.54.jar:na]

at com.jcraft.jsch.Session.connect(Session.java:183) ~[jsch-0.1.54.jar:na]

at org.apache.nifi.processors.standard.util.SFTPTransfer.getChannel(SFTPTransfer.java:433) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]

... 16 common frames omitted

My Configuration

I have tried to test the conn from nifi server with following command

sftp -o "IdentityFile=/etc/nifi-resources/keys/private_key_DWU_20140909.ppk" -oPort=2222 wftpb086@147.29.151.71

but there is no connection.

I am not sure if it is my configuration in nifi or it is a firewall issue can anyone verify my nifi configuration.

the private key path refers to my /etc/nifi-resources/keys/private_key_DWU_20140909.ppk file

[#2 · Accepted solution]

@Simon Jespersen

If you cannot get this to work outside of NiFi, it is not going to work inside of NiFi either.

But looking over your statement above, I see a couple things...

1. You are trying to use a "ppk" file. This is a Putty Private Key which is not going to be supported by SFTP. You should be using a private key in pem format.

2. SSH is very particular about permissions set on private keys. SSH will reject the key if the permissions are to open. Once you have you pem key make a copy of it for your NiFi application and make sure that copy is owned by the user running NiFi. The permissions also must be 600 on the private key.

nifi.root 770 (-rwxrwx---) will not be accepted by SSH

nifi.root 600 (-rw-------) will be accepted.

You can't grant groups access to your private key.

Thanks,

Matt

[#3]

@Simon Jespersen

Try using -vvv on your sftp command outside of NIFi to get more detail on why it is not working:

sftp -vvv -i "IdentityFile=/etc/nifi-resources/keys/<private_key.pem>" -oPort=2222 wftpb086@147.29.151.71

Matt

[#4]

Thank you very much Matt

[#5]

Hi,

I struggle several hours because everything was working fine from terminal on user nifi using my private key file with strictchecking.

However, the problem within nifi was that it use the JSch library that does not support sftp server using ed25519 protocol !

Solution was : change sftp server configuration or use login/password.

PS: thanks to nifi dev team, I love the tool

Regards

Julien

[#6]

@julien laurenceau

what do you mean change sftp server configuration ? I am having the same issue

[#7]

Hi,

There is currently no solution other than to swith back to rsa authent. In addition this library won't work if you try to use rsa with custom key length (I tried 4096 and it failed).

You have to edit the ssh_config on the sftp server in order to disable ed25519 (or at least set rsa with a higher priority)

I invite you to watch the ticket on nifi's JIRA https://issues.apache.org/jira/browse/NIFI-5816

Regards