Community Articles

Find and share helpful community-sourced technical articles.
Celebrating as our community reaches 100,000 members! Thank you!
Labels (1)
Expert Contributor

Nimbus metrics are critical to operations as well as development teams for monitoring the performance and stability of Storm applications / topology. Usually most production environments have a metrics / operations monitoring systems including solr, elasticsearch, tsdbs etc. This post shows you; how you can use Collectd to forward these metrics over to your desired metrics environment and alert on them.


Collectd is a standard metrics collection tool that can be run natively on linux operating systems. It's capable of capturing a wide variety of metrics, you can find more information on Collectd here:

So to capture Storm nimbus metrics, here's a collectd plugin that needs to be complied and built: (using Maven). Simply run:
mvn clean package assembly:single

In addition, you will need to install collectd and ensure that it has Java plugin capability. Here's a great post on how to do that: (Please note that the JAR="/path/to/jar" JAVAC="/path/to/javac" variables need to be fixed before you can run it)

Once installed, you will need to configure collectd using the following: (DON'T FORGET TO CONFIGURE OUTPUT PLUGIN)

LoadPlugin java

<Plugin "java">  
# required JVM argument is the classpath  
# JVMArg "-Djava.class.path=/installpath/collectd/share/collectd/java" 
# Since version 4.8.4 (commit c983405) the API and GenericJMX plugin are 
# provided as .jar files.  

JVMARG "-Djava.class.path=<ABSOLUTE PATH>/lib/collectd-api.jar:<ABSOLUTE PATH>/target/storm-collectd-0.0.1-SNAPSHOT-jar-with-dependencies.jar"  

LoadPlugin "com.srotya.collectd.storm.StormNimbusMetrics"

<Plugin "storm">
	address "http://localhost:8084/"
	kerberos false	
	jaas "<PATH TO JAAS CONF>"  

0 Kudos