Where does CDH5 store the list of data nodes?

[#1]

I have built and existing demo cluster. It was working perfectly well.

however because of some changes, I have to change the IP address of each of the data nodes.

I did a grep -R 'oldIP' /etc on each machine and edited the files which contained the old IP addresses and replaced them with new IP.

I rebooted each machine.

However despite doing that when I do

sudo -u hdfs hadoop dfsadmin -report

it shows me 2 dead data nodes and it lists the old IP addresses.

How can I remove old IP and then replace them with new IP addresses?

[#2]

This is the error. 

CRITICAL Initialization failed for Block pool BP-1219478626-192.168.1.20-1418484473049 (Datanode Uuid null) service to nn1home/10.192.128.227:8022 Datanode denied communication with namenode because the host is not in the include-list: DatanodeRegistration(10.192.128.231, datanodeUuid=ff6a2644-3140-4451-a59f-496478a000d7, infoPort=50075, ipcPort=50020, storageInfo=lv=-56;cid=cluster18;nsid=850143528;c=0) at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.registerDatanode(DatanodeManager.java:889) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FSNamesystem.java:4798) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.registerDatanode(NameNodeRpcServer.java:1037) at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.registerDatanode(DatanodeProtocolServerSideTranslatorPB.java:92) at org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:26378) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:587) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)

[#3]

Are you using cloudera cm? you have to remove the old ip from the hadoop config and add the new ones.

[#4]

hadoop dfsadmin -refreshNodes

will help you regenerate the datanode list....

[#5]

If you use Cloudera Manager, you can look at the list of hosts and try
to recomission any nodes.

If not, you need to look at the file pointed to by the dfs.hosts
property and ensure the new IP address or host name is listed in it.
Then as TGrayson mentioned, you should run sudo -u hdfs hadoop
dfsadmin -refreshNodes"

Let us know if it helped.

[#6 · Accepted solution]

this command did not solve the issue. 

So I deleted my cluster and rebuilt it using the right IP addresses.