I have one multinode HDP3.1.5 cluster and As per given in the   Apache Hadoop 3.1.1 Documentation   , it is expected to show all the metrics including   NumTransactionsBatchedInSync60sNumOps   and   NumTransactionsBatchedInSync60s50thPercentileLatencyMicros   on Journal nodes REST response given on 8480 port, these metrics are missing in REST responses of Journal Node. I even tried to configure   dfs.metrics.percentiles.intervals   property in hdfs-site.xml, but instead of exposing all metrics on journal node REST, the metrics are exposed on Namenodes REST response given on port 50070. Could you please help me how to expose such metrics on Journal Node as well? Is there any other journal node specific property which I need to configure to expose such metrics? ... View more

I need to check the status of ZKFailOverController(ZKFC) service whether it is running or not, is there any way to collect the ZKFC health/ monitoring metrics in java code easily?   Also does the dfs.ha.zkfc.port property is version specific? I had one observation that this property is visible automatically on HDP 3.1 but not on HDP 2.5 although it is HA enable. ... View more

I want to fetch the config parameters from the conf REST response, but not sure whether the solution of fetching config details from http://hostname:50070/conf worked or not for all types of clusters which are having the secured endpoints. Please assist here. ... View more

Looking for the Journal port in any JMX response except http://hostname:50070/conf, Also searching for the List of active journal nodes Currently i'm getting the list of journal nodes from "NameJournalStatus" : "[{\"manager\":\"QJM to ["host1:8485","host2:8485","host3:8485"]" but its not clear whether this is list is of active journal nodes or not and also the port 8485 is IPC port but I want JMX port like default ports 8480 or 8481 of journal node ... View more

BlockCapacity given in the REST reponse of Namenode is different than what we have set in the Ambari under HDFS dfs.blocksize On Rest I am getting the value for BlockCapacity is 2097152 which equivalent to 2mb although the value of blocksize in Ambari is 128mb by default Fetching the blockcapacity from from following REST url   https://<namenode-ip>:50470/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem Does BlockCapacity from namenodes rest api and dfs.blocksize from Ambari are the same or different?   ... View more

Response of Jobs API : http://<history server http address:port>/ws/v1/history/mapreduce/jobs { "jobs": { "job": [{ "submitTime": 1574157410876, "startTime": 1574157419359, "finishTime": 1574157459602, "id": "job_1568869492319_0009", "name": "word count", "queue": "default", "user": "hdfs", "state": "FAILED", "mapsTotal": 0, "mapsCompleted": 0, "reducesTotal": 0, "reducesCompleted": 0 }] } }   Response of Job API : http://<history server http address:port>/ws/v1/history/mapreduce/jobs/{jobid} { "job": { "submitTime": 1574157410876, "startTime": 1574157419359, "finishTime": 1574157459602, "id": "job_1568869492319_0009", "name": "word count", "queue": "default", "user": "hdfs", "state": "FAILED", "mapsTotal": 9, "mapsCompleted": 0, "reducesTotal": 1, "reducesCompleted": 0, "uberized": false, "diagnostics": "Task failed task_1568869492319_0009_m_000006\nJob failed as tasks failed. failedMaps:1 failedReduces:0", "avgMapTime": 0, "avgReduceTime": 0, "avgShuffleTime": 0, "avgMergeTime": 0, "failedReduceAttempts": 0, "killedReduceAttempts": 0, "successfulReduceAttempts": 0, "failedMapAttempts": 17, "killedMapAttempts": 7, "successfulMapAttempts": 0 } }   See the difference in the mapsTotal and reducesTotal values, Could anybody have idea whats happening here? I'm running the jobs on HDP3.0 platform.   ... View more