Adding a Service RPC Port to an Existing HA Cluster with ZKFCs
Configuring a separate service RPC port can improve the responsiveness of the NameNode by allowing DataNode and client requests to be processed via separate RPC queues.
Adding a service RPC port to an HA cluster with automatic failover via ZKFCs requires a workaround step.
The steps to configure a service RPC port are as follows:
- Add the following settings to hdfs-site.xml.
<property>
<name>dfs.namenode.servicerpc-address.mycluster.nn1</name>
<value>nn1.example.com:8040</value>
</property>
<property>
<name>dfs.namenode.servicerpc-address.mycluster.nn2</name>
<value>nn2.example.com:8040</value>
</property>
2. Restart NameNodes.
3. Stop the ZKFC processes on both NameNodes
4. Run the following command to reset the ZKFC state in ZooKeeper
hdfs zkfc -formatZK
Without this step you will see the following exception after ZKFC restart.
java.lang.RuntimeException: Mismatched address stored in ZK for NameNode
5. Restart the ZKFCs.

