Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

How to check disks health in Hadoop clusters and Kafka machines

We have many Hadoop clusters. Each datanode holds many disks (between 10 and 20).

 

All disks are used for HDFS. We use well-known tools like smartctl to check the disks.

 

We want to know if there is a dedicated tool to check disks that hold HDFS file-system.

 

We are interested in a similar tool for Kafka disks (since Kafka disks are very important).

Michael-Bronson
1 REPLY 1

Expert Contributor

Hi @mike_bronson7 

 

There is no generic/specific tool to monitor HDFS/KAFKA disk.Most client generally use the tool which is incorporated with OS vendor or either opt for thrid party tools. However there are multiple external tools available through which you can achieve this task -

 

Nagios / OpsView /HPOMI are popular options I've seen.

In our environment KAFKA is extensively used and we have HPOMI and Prometheus installed for monitoring.