Apache NiFi 2.6.0/2.7.2 ListSFTP Hostname not working when using Proxy Configuration Service.

[#1]

Apache NiFi 2.6.0/2.7.2 ListSFTP Hostname not working when using Proxy Configuration Service.   It works when an IP address is listed.

Example:
Hostname: cloudera.com
Port: 22
Proxy Configuration Service: StandardProxyConfigurationService (Proxy Type=HTTP)

When executed (run once), it gets the following error:
ListSFTP[id=a423ae96-019b-1000-c5c2-8e609a342bc5] Processing failed: org.apache.nifi.processors.standard.socket.ClientConnectException: SSH Connection failed [cloudera.com:22]
- Caused by: org.apache.sshd.common.SshException: DefaultConnectFuture[A@cloudera.com/<unresolved>:22]: Failed (UnknownHostException) to execute: cloudera.com
- Caused by: java.net.UnknownHostException: cloudera.com

When using the IP address of cloudera.com as 151.101.127.10 will work (disregard the proxy 403 error as it is not white listed) 
ListSFTP[id=a423ae96-019b-1000-c5c2-8e609a342bc5] Processing failed: org.apache.nifi.processors.standard.socket.ClientAuthenticationException: SSH Authentication failed [151.101.127.10:22]
- Caused by: org.apache.sshd.common.SshException: [ssh-connection]: Failed (HttpProxyConnectException) to execute: http, none, proxy.*****.**/xxx.xx.xx.xxx:8080 => /151.101.127.10:22, status: 403 Forbidden
- Caused by: io.netty.handler.proxy.HttpProxyHandler$HttpProxyConnectException: http, none, proxy.*****.**/xxx.xx.xx.xxx:8080 => /151.101.127.10:22, status: 403 Forbidden

When the above is executed under NiFi 1.26.0 works fine with hostname or IP address when using the proxy.

Anyone have seen this issue?

Please advise.

[#2]

@NSX, Welcome to our community! To help you get the best possible answer, I have tagged in our NiFi experts @MattWho @haridjh @vats  who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

[#3]

@NSX 

Caused by: java.net.UnknownHostException: cloudera.com

Above is telling you that your Apache NIFi 2.6.0 /2.7.2 server is unable to resolve "cloudera.com to an IP address.  You Apache NiFi 1.26 must be successful in hostname resolution and thus working.

What if you manually added following to your hosts file on your 2.7.2 servers?
151.101.127.10 cloudera.com

Have you tried pinging cloudera.com from both your 2.7.2 and 1.26 servers?

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

[#4]

@MattWho 

Server under 1.26.0 and 2.7.2 are behind the firewall. Both servers were in the same subnet.  After we upgraded one of the server from 1.26.0 to 2.6.0 then 2.7.2 having issue using the Proxy Configuration service (Proxy server) to reach the internet ie: cloudera.com.

$ curl -x http://proxy.xxxx.ca:8080 sftp://cloudera.com:22 -v
* Trying xxx.xx.xx.xxx:8080...
* Connected to proxy.xxxx.ca (xxx.xx.xx.xxx) port 8080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to cloudera.com:22
> CONNECT cloudera.com:22 HTTP/1.1
> Host: cloudera.com:22
> User-Agent: curl/7.76.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 403 Forbidden
< Cache-Control: no-cache
< X-XSS-Protection: 1
< Connection: Keep-Alive
< Content-Type: text/html; charset=utf-8
< Content-Length: 729
< Pragma: no-cache
<
* Received HTTP code 403 from proxy after CONNECT
* CONNECT phase completed!
* Closing connection 0
curl: (56) Received HTTP code 403 from proxy after CONNECT

Disregard the proxy 403 error as it cloudera.com is not listed in the proxy white list.

The OS curl connection is working so it is not related to the host file (we cant use it behind the firewall).   In NiFi 1.26.0 works fine.  The server upgraded to 2.6.0/2.7.2 was 1.26.0 previously and didnt have this issue.  Seems to be a bug with 2.x. 

If we use IP address in NiFi, the ListSFTP will work but not when using the hostname ie: cloudera.com; seems to be an issue passing the hostname to the proxy.

DefaultConnectFuture[A@cloudera.com/<unresolved>:22]: Failed (UnknownHostException) to execute: cloudera.com

Please advise.