I would like to know how to obtain the values of total_io_mb, mapreduce_inputBytes, and mapreduce_outputBytes for each application within a certain period of time. I believe that these values are written to the yarn log file. So, I would like to know SME's advice on how to do this, that is, how to configure these values to be written to the log. By Cloudera Manager, we are not able to get but the rate(ex. yarn_application_hdfs_bytes_read_rate). However, alternative ideas are welcome. The purpose is to evaluate the disk IO of the entire CDH cluster, which is made up of several hundred bare metal machines, and to estimate the disk IO required for the next generation of bare metal and/or storage nodes.
... View more
