Monitor Hadoop JVMs with jVisualVM
The goal of this article is to provide you step by step instruction to install jVisualVM to monitor JVMs inside your hadoop environment. Consult your operations team prior to making any production changes
Go here to download jVisualVM - tool to visually monitor JVM in hadoop. HBase is a great example where you want to visually analyze JVM health. You may already have this tool on your workstation. Simple go to command line and type jVisualVM. If it comes up your in business. else download it:
Once you have application up and running install the Visual GC plugin. Tools ->available plugins->Visual GC
Lets go to the node you want to monitor. For jVisualVM to work it needs jstatd to run on the node. Once jstatd is running it automatically sends updates of remote applications running on the node to jVisualVM.
Run jstatd from command line.
If you get the following error:
We need to perform an additional simple configuration. Do the following steps
Run which jstatd then cd into the bin location
Create a policy file jstatd.all.policy in the bin directory (same location as jstatd). We are building a policy to allow jstatd to have permission to everything.
Inside the file add the following:
grant codebase "file:${java.home}/../lib/tools.jar" {
permission java.security.AllPermission;
};
Now run following command to start jstatd with your policy:
./jstatd -J-Djava.security.policy=jstatd.all.policy &
End of steps for jstatd error
Test if jstatd is running by issuing this command:
jps -l -m -v rmi://localhost
You will see tons of output such as:
Which looks good
Open jvisualVM (command line type jVisualVM)
Right click on remote and select add remote host
Add hostname or IP address of node you want to analyze:
Now you are connected. Click on arrow next to your remote. This will display all JVMs running on the node:
Now you can start analyzing. lets click on hbase data node and click on monitor tab:
Click on Visual GC tab to analyze different generations inside the JVM.
I hope this article helps you start analyzing at a deeper level JVMs on hadoop especially with hbase.
