Skip to main content
sunile_manjee
Gold ⭐️
sunile_manjee
Gold ⭐️
October 12, 2016

Enable JMX metrics on hadoop using jmxterm

  • October 5, 2026 at 9:35 AM
  • 3 replies
  • 9441 views
  • Translate All

To start pulling jmx metrics from hadoop you need to first enable it via jvm parameters. Go to ambari-->Yarn-->config-->yarn-env

Enable jmx by adding the following params to YARN_RESOURCEMANAGER_OPTS

"-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=8001"

Set the port to whatever port is available on your cluster

8473-2016-10-12-10-25-38.jpg

Save the new config. This will require restart of yarn.

Now you have jmx enabled but you need to client to start pulling jmx metrics. Go to your data node and and download (from here http://wiki.cyclopsgroup.org/jmxterm/download.html) the latest jxmterm-xxx-xxx-uber.jar. For this article I use jmxterm-1.0-alpha-4-uber.jar

Once you have the jmxterm client downloaded lets connect to jxm using

java -jar jmxterm-1.0-alpha-4-uber.jar -l service:jmx:rmi:///jndi/rmi://localhost:<YOURPORT>/jmxrmi

In this example I am set port to 8012.

java -jar jmxterm-1.0-alpha-4-uber.jar -l service:jmx:rmi:///jndi/rmi://localhost:8012/jmxrmi

8474-2016-10-12-10-32-44.jpg

Now I am connected to jxm. Lets look at all the beans available by issuing beans commands

8475-3.jpg

Well now I see all the beans available to pull metrics from. Lets say I want to pull metrics about resourcemanager cluster metrics. That is bean Hadoop:name=ClusterMetrics,service=ResourceManager. Lets find all the attributes available for that beans by issuing info -b Hadoop:name=ClusterMetrics,service=ResourceManager

8476-4.jpg

All the attributes are shown and notice there is a notification attribute. You can use this for notification into your enterprise operational system. To pull metrics for a specific attribute within a bean use

get -b bean_name attribute

For this example I want to know number of active Node Managers which is attribute NumActiveNMs.

get -b Hadoop:name=ClusterMetrics,service=ResourceManager NumActiveNMs

8477-5.jpg

So there are 4 active node managers. I want to know how many node managers are down. That is attribute NumLostNMs

get -b Hadoop:name=ClusterMetrics,service=ResourceManager NumLostNMs

8478-6.jpg

Returns 0 meaning all my node managers are available.

Hope this helps you get started on interacting with JMX.

    3 replies

    ScipioTheElder
    ScipioTheElder
    Gold ⭐️
    October 14, 2016

    Great article. Is there any way to pull individual yarn container JMX? It would be great for me to see what is going on when I start a job and monitor the job containers at real time.

    Translate
    sunile_manjee
    Gold ⭐️
    sunile_manjeeAuthor
    Gold ⭐️
    October 14, 2016

    @ScipioTheYounger most definitely. Simply use exact same process and connect to yarn jvm. you will get all attributes (metrics) available.

    Translate
    ScipioTheElder
    ScipioTheElder
    Gold ⭐️
    October 17, 2016

    Sunile.

    For daemons such as NM and RM, I can hard-code JMS port number. However, how to set different JMX port number on a number of JVM containers running on the same slave node? For example, I can configure 18 container per node, how could I set the port number to 8001-8018? Maybe I can use ZK, or some kind of script.

    Translate
    Terms & ConditionsAccessibility statement