This is a great flow to use for performance benchmarking and analysis. Very simple. How fast is it? How fast should it be? Where are the bottlenecks? What can we tune? What can we design better?711dd39b-0157-1000-833f-e4137b7d82e7Pure Split Merge01571001-e1b8-111f-0000-000000000000711dd39b-0157-1000-0000-0000000000002 GB2000711dd39b-0157-1000-0000-00000000000001571000-e1b8-111f-0000-000000000000PROCESSOR0 sec1splits711dd39b-0157-1000-0000-000000000000711fe1b8-0157-1000-0000-000000000000PROCESSOR001571007-e1b8-111f-0000-000000000000711dd39b-0157-1000-0000-0000000000002 GB10000740.4444580078125381.6355579322295738.5072631835938156.91614417490527926.419189453125155.94753150400683711dd39b-0157-1000-0000-00000000000001571017-e1b8-111f-0000-000000000000PROCESSOR0 sec2splits711dd39b-0157-1000-0000-00000000000001571000-e1b8-111f-0000-000000000000PROCESSOR00157101e-e1b8-111f-0000-000000000000711dd39b-0157-1000-0000-0000000000001 GB500711dd39b-0157-1000-0000-00000000000001571004-e1b8-111f-0000-000000000000PROCESSOR0 sec0merged711dd39b-0157-1000-0000-00000000000001571017-e1b8-111f-0000-000000000000PROCESSOR0711f3c21-0157-1000-0000-000000000000711dd39b-0157-1000-0000-0000000000001 GB1711dd39b-0157-1000-0000-000000000000711f0b58-0157-1000-0000-000000000000PROCESSOR0 sec1success711dd39b-0157-1000-0000-000000000000711e5819-0157-1000-0000-000000000000PROCESSOR071207556-0157-1000-0000-000000000000711dd39b-0157-1000-0000-0000000000002 GB1363.65203857421875333.2046497291045361.7147827148437595.8932025855498547.68945312596.86182288584277711dd39b-0157-1000-0000-000000000000711fe1b8-0157-1000-0000-000000000000PROCESSOR0 sec2success711dd39b-0157-1000-0000-000000000000711f0b58-0157-1000-0000-000000000000PROCESSOR001571000-5ec3-1639-0000-000000000000711dd39b-0157-1000-0000-000000000000465.356969572031417.4351347484456543.58781814575195175.319778442382801571002-5ec3-1639-0000-000000000000711dd39b-0157-1000-0000-000000000000848.929791281608286.207024637103343.58781814575195196.6293640136718876395ec3-0157-1000-0000-000000000000711dd39b-0157-1000-0000-000000000000104.06237528803490.040.681968688964844129.794799804687501571000-e1b8-111f-0000-000000000000711dd39b-0157-1000-0000-000000000000376.94757080078125320.1170947974639WARN1Line Split CountLine Split CountMaximum Fragment SizeMaximum Fragment SizeHeader Line CountHeader Line CountHeader Line Marker CharactersHeader Line Marker CharactersRemove Trailing NewlinesRemove Trailing Newlinesfalse30 secLine Split Count1Maximum Fragment SizeHeader Line Count0Header Line Marker CharactersRemove Trailing Newlinesfalse00 secTIMER_DRIVEN1 secsSingle Linestruefailuretrueoriginalfalsesplitsorg.apache.nifi.processors.standard.SplitText01571004-e1b8-111f-0000-000000000000711dd39b-0157-1000-0000-000000000000756.7015392823873388.4301613050585WARN1Merge StrategyMerge StrategyMerge FormatMerge FormatAttribute StrategyAttribute StrategyCorrelation Attribute NameCorrelation Attribute NameMinimum Number of EntriesMinimum Number of EntriesMaximum Number of EntriesMaximum Number of EntriesMinimum Group SizeMinimum Group SizeMaximum Group SizeMaximum Group SizeMax Bin AgeMax Bin AgeMaximum number of BinsMaximum number of BinsDelimiter StrategyDelimiter StrategyHeader FileHeader FileFooter FileFooter FileDemarcator FileDemarcator FileCompression LevelCompression LevelKeep PathKeep Pathfalse30 secMerge StrategyBin-Packing AlgorithmMerge FormatBinary ConcatenationAttribute StrategyKeep Only Common AttributesCorrelation Attribute NameMinimum Number of Entries1500Maximum Number of EntriesMinimum Group Size1500 MBMaximum Group SizeMax Bin Age60 secsMaximum number of Bins100Delimiter StrategyFilenameHeader FileFooter FileDemarcator FileCompression Level1Keep Pathfalse00 secTIMER_DRIVEN1 secsMerge Every 60 Secstruefailuretruemergedtrueoriginalorg.apache.nifi.processors.standard.MergeContent01571017-e1b8-111f-0000-000000000000711dd39b-0157-1000-0000-000000000000754.4768979874402195.66143301771908WARN1Merge StrategyMerge StrategyMerge FormatMerge FormatAttribute StrategyAttribute StrategyCorrelation Attribute NameCorrelation Attribute NameMinimum Number of EntriesMinimum Number of EntriesMaximum Number of EntriesMaximum Number of EntriesMinimum Group SizeMinimum Group SizeMaximum Group SizeMaximum Group SizeMax Bin AgeMax Bin AgeMaximum number of BinsMaximum number of BinsDelimiter StrategyDelimiter StrategyHeader FileHeader FileFooter FileFooter FileDemarcator FileDemarcator FileCompression LevelCompression LevelKeep PathKeep Pathfalse30 secMerge StrategyBin-Packing AlgorithmMerge FormatBinary ConcatenationAttribute StrategyKeep Only Common AttributesCorrelation Attribute NameMinimum Number of Entries4000Maximum Number of Entries5000Minimum Group Size0 BMaximum Group SizeMax Bin Age5 secsMaximum number of Bins100Delimiter StrategyFilenameHeader FileFooter FileDemarcator FileCompression Level1Keep Pathfalse00 secTIMER_DRIVEN1 secsMerge 5K Chunkstruefailurefalsemergedtrueoriginalorg.apache.nifi.processors.standard.MergeContent711e5819-0157-1000-0000-000000000000711dd39b-0157-1000-0000-0000000000000.064.48306159433886WARN1Input DirectoryInput DirectoryFile FilterFile FilterPath FilterPath FilterBatch SizeBatch SizeKeep Source FileKeep Source FileRecurse SubdirectoriesRecurse SubdirectoriesPolling IntervalPolling IntervalIgnore Hidden FilesIgnore Hidden FilesMinimum File AgeMinimum File AgeMaximum File AgeMaximum File AgeMinimum File SizeMinimum File SizeMaximum File SizeMaximum File Sizefalse30 secInput Directory/Users/jwitt/Downloads/test-dataFile Filter[^\.].*Path FilterBatch Size10Keep Source FiletrueRecurse SubdirectoriestruePolling Interval0 secIgnore Hidden FilestrueMinimum File Age0 secMaximum File AgeMinimum File Size0 BMaximum File Size01 secTIMER_DRIVEN1 secGrab: Keep Originalfalsesuccessorg.apache.nifi.processors.standard.GetFile711f0b58-0157-1000-0000-000000000000711dd39b-0157-1000-0000-0000000000000.11590576171875262.7112415259795WARN1ModeModeCompression FormatCompression FormatCompression LevelCompression LevelUpdate FilenameUpdate Filenamefalse30 secModedecompressCompression FormatgzipCompression Level1Update Filenamefalse00 secTIMER_DRIVEN1 secsDecompress if neededtruefailurefalsesuccessorg.apache.nifi.processors.standard.CompressContent711fe1b8-0157-1000-0000-000000000000711dd39b-0157-1000-0000-000000000000370.95037841796875131.1379077857451WARN1Line Split CountLine Split CountMaximum Fragment SizeMaximum Fragment SizeHeader Line CountHeader Line CountHeader Line Marker CharactersHeader Line Marker CharactersRemove Trailing NewlinesRemove Trailing Newlinesfalse30 secLine Split Count5000Maximum Fragment SizeHeader Line Count0Header Line Marker CharactersRemove Trailing Newlinesfalse00 secTIMER_DRIVEN1 secs5K Line Chunkstruefailuretrueoriginalfalsesplitsorg.apache.nifi.processors.standard.SplitText09/29/2016 10:01:44 EDT