It should be in nifi-app.log... in the code it does: context.getBulletinRepository().addBulletin(bulletin); logger.warn(message); The logger is a standard SLF4J logger which ends up being logback controlled by the logback.xml in the conf directory: Logger logger = LoggerFactory.getLogger(MonitorDiskUsage.class); ... View more

In the case of SplitText the approach when splitting large files is to use two instances of SplitText, where the first one might split to 10-20k lines per flow file, and then the second splits down to 1 line. This avoids producing millions of flow files in one execution of the processor. For some other processors it is common for their description to include a warning statement if the processor is going to read in the whole flow file into memory so that the user is aware if they send in 2GB of data, its going to use 2GB of the heap, or create an OOM if its not available. Most processors whenever possible should perform their processing in a streaming fashion to avoid taking up large chunks of memory. As far as sharing the cluster among teams, NiFi doesn't really have resource isolation, but NiFi 1.0.0 (initial BETA released yesterday) is going to introduce is fine grained security model so that different teams and people can be granted access to different parts of the flow. Team1 might only have access to Process Group 1, and Team 2 might only have access to Process Group 2, so each team can't see what the other team is doing or change their flow. ... View more

Counters are a way for a processor to track how many times some event occurred, mostly for monitoring purposes. There is a method in the ProcessSession: void adjustCounter(String name, long delta, boolean immediate); So calling this method with ("myCounter", 1, true) would increment the count of "myCounter" by 1, or create the counter if it didn't exist. Counters are not persistent and will be reset on restart. An example is in the syslog processors which increment a counter for each syslog message received. ... View more

The MergeContent processor can be used to merge JSON together and has a property called "Correlation Attribute Name" which when specified will merge together flow files that have the same value for the attribute specified. In your scenario you first need to use EvaluateJSONPath to extract "service" and "eventName" from the JSON document. Based on your sample JSON it seems like they are at the root level of the document so I believe something like: service = $.service eventName = $.eventName Then you need to get these two values into a single attribute, so you can use UpdateAttribute with something like: serviceEventName = ${service}/${eventName} Then in MergeContent set the "Correlation Attribute Name" to "serviceEventName". You can also specify the minimum group size and age so that you can merge together either 100MB or 1 hour worth of data. ... View more