Apache Nifi - What are Counters in Nifi?

[#1]

Hi,

I noticed 'Counter' button on top right of Nifi UI (Counters button is right next to Summary button on Admin options).

I tried to search for any documentation however could find it. Could you guys share any link/documentations (or comments) on how to use this feature, purpose of this feature etc? Seems like an option to maintain counts, however not sure how to use it.

Thanks

[#2 · Accepted solution]

Counters are a way for a processor to track how many times some event occurred, mostly for monitoring purposes. There is a method in the ProcessSession:

void adjustCounter(String name, long delta, boolean immediate);

So calling this method with ("myCounter", 1, true) would increment the count of "myCounter" by 1, or create the counter if it didn't exist. Counters are not persistent and will be reset on restart. An example is in the syslog processors which increment a counter for each syslog message received.

[#3]

Awesome,

So by saying 'reset on restart' you mean restarting the Workflow (Process Groupe/Processor) or restarting the Nifi Instance?

- Regarding usage, do you guys use it for windowing operations, like count x every minute?

- If I am counting an attribute X in a cluster, will it show me aggregated results on the Counter dashboard (like showing aggregated value of X for the whole cluster, or show value of X for each slave)

Thanks

[#4]

By "reset on restart" I meant that they are held in memory so if the NiFi Java process restarts the counters are reset. Starting/stopping components on the graph do not impact the counters.

We don't really do windowing operations... Counters are usually just some processor specific count that could be helpful for debugging/monitoring purposes. It is really just meant for someone to look at in the UI to figure out how something is working, but not really for the processor to retrieve the value later. In fact, I don't think there is any other processor API call besides adjustCounter, so all you can really do is increment.

In a cluster I believe you should see the aggregated value of X for the whole cluster, it doesn't break it out for each node.

One other point I forgot, is that behind the scenes it automatically keeps track of the aggregate count across instances of the same type of processor, and also for each instance. So if you had two ListenSyslog processors, you should see Messages Received for All Listen Syslog Processors, Messages Received for ListenSyslog #1, and Messages Received for ListenSyslog #2.

[#5]

Hi,

Could you please let me know how the resetting of target signal count in wait processor work? Refer my question:

https://community.hortonworks.com/questions/138762/reset-of-target-signal-count-in-wait-processor.ht...

Thanks,

John