@Kilynn  The following property in the nifi.properties file controls when a swap file is created per connection. nifi.queue.swap.threshold=20000 This is per connection and not for all FlowFiles across all connections.  A FlowFile swap file will always consist of 10000 FlowFiles. So if a connection reaches 20000 queued FlowFiles, a swap file will be created for 10000 of those.  So if a connection queue reaches 40000, you would have 3 swap files of that connection. You can control individual connection queues by setting the "Back pressure Object Threshold" on a connection: Note: Threshold settings are soft limits And default for object threshold is 10000.   So with these settings there should be very little to no swapping of FlowFiles to disk happening at all.   Swap files would only happen if source processor to that connection output enough FlowFiles to connection at one time to trigger a swap file. For example: - Connection has 9000 queued FlowFiles, so back pressure is not being applied. - Source processor is thus allowed to execute - Source processor upon execution produces 12000 FlowFiles - now downstream connection has 21000 queued FlowFiles.  One swap file is produced and back pressure is enabled until queue drops back below 10000 queued FlowFiles. FlowFiles consist of two parts (FlowFile attributes/metadata and FlowFileContent).  The only portion of a FlowFile held in heap memory is the FlowFile attributes/Metadata.  FlowFile content is never held in memory (Some processors may load content in to memory in order to perform their function only). FlowFile attributes/metadata is persisted to the flowfile repository and FlowFile content is written to the content repository.  This important to avoid data loss if NiFi dies or is restarted while data still exists in connection queues. If you found this helped with your query, please take a moment to login and click accept in the solutions that helped. Thank you, Matt ... View more

In this case per the admin guide you have to also copy the users.xml and authorizations.xml to the new node.  For some reason these were not being created accurately on instantiation. ... View more

Well, I didn't RATW's... My issue was NOT that my Initial Admin was wrong in the authorizers.xml file.  It was that my nodes weren't specified with a space between the , and the OU.   Make sure your authorizers.xml is EXACTLY as is created when you create your certificates... if you have issues stop NIFI fix the issue, delete users.xml and authorizations.xml and then restart NIFI.   Nothing to see here move along. ... View more

In this case the problem isn't escaping the special characters in the middle of the flowfile, instead an efficient use of the ReplaceText processor was needed.  As it turns out the only real need was to add a comma after the last '}' of every line except the last.  ReplaceText handles this nicely and gives me the opportunity to inspect the objects line-by-line instead of the entire text.  This is accomplished with setting the Search Value: }$ (this means find a } but only if it's at the end of the line) and then setting the Replacement Value: }, replacing the } character with another but appending a comma to the end.   By inspecting line by line it also reduces resources required.     ... View more

At the end of the day I used Steve Matison and 'PVVK's solutions.  First I used PVVK's to modify my flowfile into valid html, then I successfully was able to use SplitJson processor to break up the flowfiles cleanly.  Then since there were multiple 'logEvent' entries in some of the records I saved off the header items as attributes and then used SteveMatison's method to break down each logEvent and added the header attributes back to it for processing.   Thanks all! ... View more