@Avish Saha The behavior you should be seeing here is that the mergeContent processor will take the first incoming FlowFile it sees and add it to bin 1. It will then continue to attempt to add additional FlowFiles to Bin 1 until either 1000 total FlowFiles have been added or the min size has reached 1 MB. Now lets say bin 1 has grown to 1000KB (just shy of 1 MB) and the next FlowFile would cause that bin to exceed the max group size of 1 MB. In this case that File would not be allowed to go into bin 1 and would be the first file to start bin 2. Now bin 1 hangs around because the min requirement of 1 MB has not been met and neither max entries or max group size has been met either. So you can see it is possible to fill all 5 of your bins without meeting your very tightly configured thresholds. So what happens when a next FlowFile will not fit in any of the 5 existing bins? The mergeContent processor will merge to oldest bin to free room to start a new bin. So what I am assuming here is you are seeing few or no files that are exactly 1 MB. Thanks, Matt ... View more

@Rohit Ravishankar How do you plan to combine the "content" of these two files together? This is the first question that needs to be addressed. You can use the mergeContent processor to merge the "content" of multiple FlowFiles using binary concatenation. If this is acceptable, then all you need to do is consume both your datafile and control file, use updateAttribute to extract the a common name from both filenames into new attribute, and finally use that new attribute as the "correlation Attribute Name" in the MergeContent processor. You would also want to set the min number of entries to 2 in the mergeContent processor. Matt ... View more

@Joshua Adeleke Also check your NiFi app log for any Out Of Memory Errors (OOME). The SplitText processor may be having memory issues trying to split over 40k records. You could try using two splitText processors in series with the first splitting on a 10,000 "Line Split Count" and the second then splitting those 10,000 line FlowFiles with a 1 "Line Split Count". This will greatly reduce the heap memory footprint. In addition, if you do a listing on the queue feeding your putSQL processor, do you see any listed FlowFiles with an unexpected size? Matt ... View more

@Narasimma varman Is your NiFi a single instance of NiFi or a NiFi cluster? If it is a cluster, keep in my mind by default the GetFile processor will be running on every node in that cluster. The validate will also run on every node as well, so make sure the directory exists on all nodes. Also make sure you have only specified a d directory path in the "Input Directory" property in GetFile. In you case, you should have only "/root/example" for that property. The filename you wish to pickup should be specified in the "File Filter" property. Matt ... View more