@mbaid You should consider opening a new thread as this one is quite old. A new thread would also allow you to provide further details that will be unique to your situation.  ... View more

@aquib_mohammed  What is the format of the files? ... View more

With that big of a file, you are going to need to adjust settings (memory) and evaluate performance of the file repositories.  I would recommend to start with small files, and work your way up.  Complete a benchmark at each stage, then begin testing config changes.  This should be a lengthy evaluation where in the end, you end up with a configuration capable of consuming 3 gb files. ... View more

Thanks @Wynner, it works! ... View more

@leah wang The NiFi java process is actually two processes. The actual NiFi java process and a bootstrap java process. The bootstrap process has a very small memory footprint, 12MB to 24MB, that looks for the NiFi java process. If it doesn't find that process it will try to start it. The NiFi java process will reserve the memory allocated, in this case 8GB, and hold it. It doesn't actually use the memory, but reserves it so it is available as NiFi needs it. There won't be any garbage clean up until the memory usage is around 80% utilization. ... View more

Kafka Consumer Demarcator could be one solution.Use Shift+enter and try persisting into HDFS. ... View more

@David Sandoval That is great to hear. Would you do me a favor and accept my answer? ... View more

@Pavan M As you are not transferring any of the flowfiles to REL_FAILURE, Transfer the else flowfiles to failure relation and auto terminate the failure relation, else: session.transfer(flowFile, REL_FAILURE) Auto terminate failure relation (or) You can use session.remove to remove the flowfile else: session.remove(flowFile) by using any of the above ways you can achieve same result as you are expecting. ... View more