@Matt Clarke Thanks Matt, very useful info. It was about 20 tar files, which turned into almost 1000 individual files that I was looking to ZIP back to 20 files. Looks like the major problem was the bin #. It was set to 1, once I increased that it had no problem with the multiple tar files that were queued up. I only had 1 concurrent tasks so I was surprised that even with 1 bin, it would look to create a new bin. For selected prioritizers it was the default " first in first out", so if its untaring one tar file at a time it should finish a whole bin before moving to the next one. ... View more

@Matt Thanks, Matt this is something i was looking at. ... View more

@Matt Thank you for your clarification. ... View more

Thanks @Wynner! ... View more

Thanks @Matt ... View more

Thanks @Matt for confirming. Sorry for not clarifying it better, my use case is to merge flow files that are from the same day, but I've both max and min # of entries set to 100, since I want to merge every 100 incoming flow files into a new merged file, as these files are small, less than 5kb each. So, I'm not trying to merge all flowfiles for the day into just 1 file. ... View more

@Matt, couple of follow up questions on Processor group with multiple input ports; 1) within the processor group, how do you distinguish between flowfiles that are coming from the various input ports. 2) in data provenance screen, is there a way to tell which flowfiles are from which input ports ... View more

Thanks, now I can load from local ... View more

@Tarek Elgamal Assuming you are referring to settings for "Max Timer Driven Thread count"? That setting controls the max number of threads that can execute at one time. Does not guarantee any order to the execution of threads. NiFi's controller in the back ground does not operate under this thread pool. Both processors will be scheduled to run based on their configured run schedule. Those concurrent tasks then get stacked in a request queue waiting on one of the threads from that pool to service them. This way, every processor is eventually going to get a chance to run thier code. Also keep in mind that some processors work on batches of FlowFiles while others process one FlowFile per task. Also hard to say that each processed FlowFile will take same amount of time to complete an operation. Really depends on processor and what it is designed to do. Thanks, Matt ... View more

@Sunile Manjee Also keep in mind that NiFi Content archiving is enabled by default with a retention period of 12 hours or 50% disk utilization before the archived content is removed/purged. The purging of FlowFile manually within your dataflow will not trigger the deltion of archived FlowFiles. ... View more