@Sravanthi Bellamkonda In order for MergeContent processor to create ~64 MB merged FLowFiles from 1 KB source FlowFiles, it would need to merge ~65,500 FlowFiles. While the MergeContent processor is merging FlowFiles in a "Bin" the FlowFile mAttributes (metadata) is being held in NiFI's JVM heap memory. This can commonly result in a Out Of Memory (OOM) condition. A more common approach is to use two MergeContent processor in series to reduce the overall heap memory footprint for such a dataflow. ListenTCP --> (success) --> MergeContent --> (merged) --> MergeContent --> (merged) --> PutHDFS The first MergeContent processor would merge based upon in your case perhaps every 1024 KB "Minimum Group Size" and "Maximum Group Size" of perhaps 1040 KB. This would roughly merge ~1,000 FlowFiles per bin. These merged FlowFiles are then passed to another MergeContent processor that will merge based on every 60 MB "Minimum Group Size" and "Maximum Group Size" of perhaps 64 MB. This will result in merging ~60 FlowFiles per Bin. I would set each "Maximum number of Bins" on both these MergeContent processors to 11. This would allow you to set the increase the "Concurrent tasks" on each MergeContent processor higher to improve performance. I would start with 3 - 5 concurrent tasks and see how that performs based on incoming data rate. I would not increase higher then 10. Just remember the more concurrent tasks given to any single processor equates to more CPU usage. So always start low and slowly increment up. Generally we try to keep the number of FlowFiles merged per processor to between 10,000 to 20,000 to minimize heap usage. Another use article about tuning NiFi's Listen based processors, can be found here: https://community.hortonworks.com/articles/30424/optimizing-performance-of-apache-nifis-network-lis.html Thanks, Matt ... View more

@Juan Manuel Nieto You cannot specify both an "Initial Admin Identity" and a "Legacy Authorized Users File" in the authorizers.xml file. Try removing the the legacy Authorized Users File and restarting NiFi to see if the users.xml and authorizations.xml files get generated. Thanks, Matt ... View more

@Juan Manuel Nieto NiFi must be configured to run securely over https using SSL before any user authentication can be used. Thanks, Matt ... View more

@sally sally You could use an UpdateAttribute and RouteOnAttribute processor to delay FlowFiles by two minutes before passing them to your next processor instead. The UpdateAttribute processor is used to create a new attribute with a value of the current epoch time in Milliseconds currentTime = ${now():toNumber()} The RouteOnAttribute then adds 2 minutes to that attribute and checks to see if it os less then or equal to the current time. If it is less then, your FlowFiles are sent to the unmatched Relationship which is looped back on the processor. The FlowFiles will continue to loop until 2 minutes have past at which time they will be routed to the matched (2mins) relationship created using teh following: 2mins = ${currentTime:plus(120000):le(${now()})} Thanks, Matt ... View more

@Jon Rodriguez Breton Are you trying to see if all attributes from both FlowFiles match exactly or is their a specific attribute from each FlowFile you want to compare? My initial thought would be to use the DetectDuplicate processor. You could write the unique attribute to the DistributedMapCache service. Then compare new FlowFiles against that stored value and deleted any duplicates. That way only the first FlowFile would get passed on. Thanks, Matt ... View more