Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to use GETFTP processor in cloud installed NIFI ?

avatar
Explorer

I am using Azure cloud in which I installed nifi, and now I want to use GETFTP processor to fetch data from ftp server. The problem is com that it is not fetching a single record from FTP server, but in local it is start queuing data within a 5 minute. Can you please suggest how to resolve this issue.

1 ACCEPTED SOLUTION

avatar
Master Guru
@Gourav Bhattacharya

GetFTP processor once triggered then it will get all the files from the directory based on your confiurations.
To control the rate of Fetching files use ListFtp processor(lists 0 byte flowfiles) instead of GetFtp processor then use Control rate processor to control the rate of flowfiles processing through the Control Rate Processor then feed the Success relation to FetchFTP processor.

Flow:

1.ListFTP
2.ControlRate //control the rate of flowfiles
3.FetchFTP

in addition refer to this link to get the data and process one flowfile at a time without using Control Rate Processor.

View solution in original post

3 REPLIES 3

avatar
Master Guru
@Gourav Bhattacharya

GetFTP processor once triggered then it will get all the files from the directory based on your confiurations.
To control the rate of Fetching files use ListFtp processor(lists 0 byte flowfiles) instead of GetFtp processor then use Control rate processor to control the rate of flowfiles processing through the Control Rate Processor then feed the Success relation to FetchFTP processor.

Flow:

1.ListFTP
2.ControlRate //control the rate of flowfiles
3.FetchFTP

in addition refer to this link to get the data and process one flowfile at a time without using Control Rate Processor.

avatar
Explorer

I have used same but I am getting error in ListFTP processor that is

java.lang.reflect.InvocationTargetException: null at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) at org.apache.nifi.controller.StandardProcessorNode.lambda$initiateStart$3(StandardProcessorNode.java:1455) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Failed to properly migrate state to State Manager at org.apache.nifi.processor.util.list.AbstractListProcessor.updateState(AbstractListProcessor.java:251) ... 15 common frames omitted

avatar
Explorer

I appreciate that you have given me a new way to fetch the file from FTP server.