Clustered NiFi of 10 nodes have been recently upgraded to v1.9.2. After that the flow/provenance/content storage is not getting cleaned and are eventually filling up and getting to 100% utilization. This will stop the cluster functioning if the affected node is primary node. As a workaround, the node has to be restarted and then the utilization(/data partition) will reduce to 10% or so. Note: The cluster is docker based. ... View more

Hi, Our nifi cluster run on 10 nodes,with an external zookeeper.We had an issue where the port 2888 was blocked for one of the zookeepers and the cluster was not responding.There are 3 zookeepers and rest 2 were still active at that time but it seemed that they were going in loop and stuck.Is there any way by which we can disconnect the zookeeper when port is blocked so that the cluster continues running? ... View more

Hi, Iam using Nifi 1.3 version and this is the flow..getsftp->putsftp->putsftp. But at times after the files are picked by getsftp,say if its 190 bytes then putsftp transfers as 0 bytes. It doesnot happen all the time but at times. Please find below the error: 2019-01-07 02:00:01,272 ERROR [Timer-Driven Process Thread-7] o.a.nifi.processors.standard.PutSFTP PutSFTP[id=XXX] PutSFTP[id=XXX] failed to process session due to org.apache.nifi.processor.exception.MissingFlowFileException: Unable to find content for FlowFile: {} org.apache.nifi.processor.exception.MissingFlowFileException: Unable to find content for FlowFile at org.apache.nifi.controller.repository.StandardProcessSession.handleContentNotFound(StandardProcessSession.java:3003) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2157) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2106) at org.apache.nifi.processors.standard.PutFileTransfer.onTrigger(PutFileTransfer.java:127) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.nifi.controller.repository.ContentNotFoundException: Could not find content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=XXX, container=default, section=490], offset=0, length=190]: Stream contained only 0 bytes but should have contained 190 at org.apache.nifi.controller.repository.io.FlowFileAccessInputStream.ensureAllContentRead(FlowFileAccessInputStream.java:49) at org.apache.nifi.controller.repository.io.FlowFileAccessInputStream.read(FlowFileAccessInputStream.java:84) at java.io.BufferedInputStream.read1(BufferedInputStream.java:284) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:636) at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:540) at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:492) at org.apache.nifi.processors.standard.util.SFTPTransfer.put(SFTPTransfer.java:543) at org.apache.nifi.processors.standard.PutFileTransfer$1.process(PutFileTransfer.java:135) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2136) ... 14 common frames omitted 2019-01-07 02:00:01,272 WARN [Timer-Driven Process Thread-7] o.a.nifi.processors.standard.PutSFTP PutSFTP[id=015b10b1-853c-1b95-d98f-915e12c396bd] Processor Administrat ively Yielded for 1 sec due to processing failure 2019-01-07 02:00:01,273 WARN [Timer-Driven Process Thread-7] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding PutSFTP[id=XXX] due to uncaught Exception: org.apache.nifi.processor.exception.MissingFlowFileException: Unable to find content for FlowFile Can you please help me in finding why the files are transfered as 0 bytes? ... View more

Hi, I have a flow where it puts 50 files in destination using putsftp processor.I need to make sure that the 50 files has reached destination.I have tried using mergecontent but since its clustered nifi with 10 nodes,its not possible.Can you please suggest a solution. ... View more

Hi, I had a running executescript processor in python which worked fine till yesterday.But suddenly its throwing error as AttributeError: type object 'java.lang.Thread' has no attribute 'State' in <script> at line number 1. Is there any issue with the processor? ... View more

Hi, I have developed a data flow in which the files that are sent to a process group1 is taken via outport port and further sent to another process group2. But when i tried to set input connection for PG1 ,the connections are not listed from the dropdown(input port of PG2). Can you please help me out? ... View more

There is a flow which would generate flowfiles,set input parameters using update attribute and then process it by execute stream command.This would however place the required files in the location as mentioned in update attribute processor.Is there any way which could fetch the processed files directly from the location so as to move the processed files into another location using another processor. ... View more