question Re: How to use GETFTP processor in cloud installed NIFI ? in Archives of Support Questions (Read Only)

How to use GETFTP processor in cloud installed NIFI ?

gourav119 — Sun, 24 Jun 2018 20:33:06 GMT

I am using Azure cloud in which I installed nifi, and now I want to use GETFTP processor to fetch data from ftp server. The problem is com that it is not fetching a single record from FTP server, but in local it is start queuing data within a 5 minute. Can you please suggest how to resolve this issue.

Re: How to use GETFTP processor in cloud installed NIFI ?

Shu_ashu — Sun, 24 Jun 2018 21:20:26 GMT

@Gourav Bhattacharya

GetFTP processor once triggered then it will get all the files from the directory based on your confiurations.
To control the rate of Fetching files use ListFtp processor(lists 0 byte flowfiles) instead of GetFtp processor then use Control rate processor to control the rate of flowfiles processing through the Control Rate Processor then feed the Success relation to FetchFTP processor.

Flow:

1.ListFTP
2.ControlRate //control the rate of flowfiles
3.FetchFTP

in addition refer to this link to get the data and process one flowfile at a time without using Control Rate Processor.

Re: How to use GETFTP processor in cloud installed NIFI ?

gourav119 — Mon, 25 Jun 2018 12:00:30 GMT

I appreciate that you have given me a new way to fetch the file from FTP server.

Re: How to use GETFTP processor in cloud installed NIFI ?

gourav119 — Fri, 29 Jun 2018 01:40:53 GMT

I have used same but I am getting error in ListFTP processor that is

java.lang.reflect.InvocationTargetException: null at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) at org.apache.nifi.controller.StandardProcessorNode.lambda$initiateStart$3(StandardProcessorNode.java:1455) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Failed to properly migrate state to State Manager at org.apache.nifi.processor.util.list.AbstractListProcessor.updateState(AbstractListProcessor.java:251) ... 15 common frames omitted