The following is under the assumption you do not explicitly require using Site to Site to transmit the data. While there are facilities that could support this, it is not an extension point and currently does not provide control over how FlowFiles are delivered. The simplest and most naive approach would be to use RouteOnAttribute to route each FlowFile to a given relationship and then use that to feed your transmission processor of choice. In this case, a PostHTTP sending to a ListenHTTP would be one way of attack that would allow transmission formatted as FlowFiles. Depending on the source system you might, such as if it was clustered, need to use expression language to additionally mark the destination system and use that to dynamically craft the resultant POST URL to the associated listener. This is fairly static and simple but would cover the use case you were anticipating. ... View more

There is not such a mechanism in place for ExecuteStreamCommand. ExecuteProcess does have the ability to batch but based on your mention of incoming event rate, seems like the need for handling input which ExecuteProcess does not provide. Depending on the nature of your parsing/process, it may be possible to convert this to use InvokeScriptedProcessor which could tie said parser/process to the component lifecycle. ... View more

@Randy Gelhausen There are a few ways to do this. Use the distributed map cache to get runtime attribute lookups and re-populate it as needed with new configs. Use a scripted processor to lookup your config values and merge the attributes on to the FlowFile. I have some work in progress extending a lookup table service by @Andrew Grande that can do lookups against a properties file that is reloaded periodically. It includes a LookupAttribute processor that can merge in either specific properties or all the properties from a properties file. See: https://github.com/jfrazee/nifi-lookup-service/tree/file-based-lookup-service ... View more