GetFtp, ListFTP, FetchFTP in cloud nifi server are not fetching the data from specific ftp server ?

[#1]

I am getting issue while working with GetFTP, ListFTP in my cloud nifi server, the configuration that I had for GetFtp is working in local nifi server but the same configuration is not working in the cloud environment. Some other FTP servers are working but that specific FTP server is not working in the cloud. Other FTP server I have which are running in passive mode but that FTP server which I want to fetch data is running on Active connection mode. I checked with the command line of that particular cloud it is working and showing connectivity. For firewall side, I disabled the firewall option. I'm not able to figure out How I will fetch data from that FTP server, please someone help me, it's very urgent.

I tried in my Logs in logs nothing is showing, no exception and errors, only GetFTP server id and some writAheadLog something is there nothing else.

Log :- 2018-06-30 10:24:18,631 INFO [NiFi Web Server-18] o.a.n.c.s.StandardProcessScheduler Starting GetFTP[id=01641000-7d8e-102b-7a2f-83073ddb9d5f] 2018-06-30 10:24:18,632 INFO [Monitor Processor Lifecycle Thread-1] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled GetFTP[id=01641000-7d8e-102b-7a2f-83073ddb9d5f] to run with 1 threads 2018-06-30 10:24:19,029 INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@587ea2b // Another save pending = false

Even I changed the nifi version also. Now I'm using nifi-1.7.0 where also it is not fetching that particular FTP server.

[#2]

When you say it is "not working", what exactly is happening?

There can only really be three possible outcomes...

a) it fetched successfully

b) it did not fetch successfully and there is a red error on the processor in the UI, and an error in nifi-app.log

c) it is stuck connecting and there is a little number icon in the top-right of the processor which shows that 1 thread is still running trying to execute

Which of these choices describes the result?

[#3]

it is not fetching a single data from ftp server that I'm using in cloud nifi server, but same configuration of FTP server in local using GetFTP processor or ListFTP processor is able to fetch data or queue data from ftp server.

If I'm using ListFTP processor in cloud nifi it is throwing an exception that is "Failed to perform listing on remote host due to java.net.SocketException", after some time (15 minutes interval). If in case I used GetFTP processor then it is not showing an exception or error message on the right corner of the nifi processor. Even in logs no error message or exception.

while using GETFTP processor log only showing: - "Log :- 2018-06-30 10:24:18,631 INFO [NiFi Web Server-18] o.a.n.c.s.StandardProcessScheduler Starting GetFTP[id=01641000-7d8e-102b-7a2f-83073ddb9d5f] 2018-06-30 10:24:18,632 INFO [Monitor Processor Lifecycle Thread-1] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled GetFTP[id=01641000-7d8e-102b-7a2f-83073ddb9d5f] to run with 1 threads 2018-06-30 10:24:19,029 INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@587ea2b // Another save pending = false"

[#4]

If ListFTP is showing an error in the UI then that error message is in nifi-app.log somewhere, please provide the full stacktrace that goes with that error.

[#5]

Let me tell the environment which i am trying.
I am using files.bcogc.ca ftp server which uses FTPS (FTP over explict SSL), When i connect using filezilla a certificate accept pop-up apperas after accepting connection gets open and able to do listing. Similarly used ftp-ssl protocol from ubuntu cli and it got opened.
But when i use in ftp nifi processor with passive mode its giving connection timeout tried with active but still same error.

Nifi vm is in Azure cloud and outgoing ports are open and some incoming ports also.

Below is the stacktrace.

-------------------------------------------------------------------------------------------------------------------------------------------------------

This is the log that I got from nifi-app-log:- 2018-07-13 15:53:40,428 ERROR [Timer-Driven Process Thread-8] o.a.nifi.processors.standard.ListFTP ListFTP[id=01641010-7d8e-102b-2639-020d0b6d6867] Failed to perform listing on remote host due to java.net.ConnectException: Connection timed out (Connection timed out): {}
java.net.ConnectException: Connection timed out (Connection timed out)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at org.apache.commons.net.SocketClient._connect(SocketClient.java:243)
        at org.apache.commons.net.SocketClient.connect(SocketClient.java:181)
        at org.apache.nifi.processors.standard.util.FTPTransfer.getClient(FTPTransfer.java:582)
        at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:227)
        at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:191)
        at org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:106)
        at org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:405)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
        at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)