Ambari Metrics per user metrics dfs.namenode.servicerpc-address config for HA Namenode

[#1]

I am trying to use Grafana's HDFS Users dashboard and to set Ambari Metrics to collect user information, I followed the instructions in this page https://cwiki.apache.org/confluence/display/AMBARI/Enabling+HDFS+per-user+Metrics+-+2.4.0.

One of the config is "dfs.namenode.servicerpc-address" and it requires specifying a namenode host. This does not work on an HA cluster where you have two namenodes.

Any recommendations on how to configure this property?

Currently it breaks the namenodes and I had to remove it.

[#2]

@Theyaa Matti

Please try these steps: 1). In Ambari, HDFS > Configs > Advanced > Custom hdfs-site, Add the following properties. Herer "My-NN-HA" is the Cluster Service name

dfs.namenode.servicerpc-address.My-NN-HA.nn1=erie1.example.com:8021
dfs.namenode.servicerpc-address.My-NN-HA.nn2=erie2.example.com:8021
ipc.8020.callqueue.impl=org.apache.hadoop.ipc.FairCallQueue
ipc.8020.backoff.enable=true
ipc.8020.scheduler.impl=org.apache.hadoop.ipc.DecayRpcScheduler
ipc.8020.scheduler.priority.levels=3
ipc.8020.decay-scheduler.backoff.responsetime.enable=true
ipc.8020.decay-scheduler.backoff.responsetime.thresholds=10,20,30

2). After the mentoined changes you might get the following error in your Logs while starting the servivce. Also the NameNodes might be moved to "StandBy" (Both)

2017-06-20 06:38:14,116 WARN  ha.EditLogTailer (EditLogTailer.java:triggerActiveLogRoll(276)) - Unable to trigger a roll of the active NN
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category JOURNAL is not supported in state standby
        at org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.checkOperation(StandbyState.java:87)
        at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.checkOperation(NameNode.java:1979)

OR

java.lang.RuntimeException: Mismatched address stored in ZK for NameNode

3). So you will need to run the following command. Stop the ZKFC processes on both NameNodes to reset the ZKFC state in ZooKeeper

hdfs zkfc -formatZK

4). Restart the ZKFC servers and then check the grafana / AMS after 2-3 minutes time..

.