Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to disable new datanode machine from data replication

how to disable new datanode machine from data replication

hi all

 

we have HDP 2.6.4 cluster with 245 workers machines

 

each worker have ( datanode and resource manager )

 

we want to add 10 new workers machines to the cluster 

 

but we want to disable the datanode machines so no data will transfer from the old datanodes to the new datanodes

 

I thinking to do maintenance mode on the new datanode , but not sure if this action is enough in order to disable the datanodes machine on the new workers

 

 

Michael-Bronson
2 REPLIES 2
Highlighted

Re: how to disable new datanode machine from data replication

Mentor

@mike_bronson7 

Once you connect the 10 new data nodes to the cluster Ambari automatically distributes the common hadoop config file i.e hdfs-site.xml,Mapred-site.xml,yarn-site.xml etc to those new nodes so they can start receiving data blocks.

 

My suggestion as a workaround would be to add these 10 new datanodes hostnames FQDN or IP (separated by a newline character) in the dfs.exclude file on the NameNode machine, edit the <HADOOP_CONF_DIR>/dfs.exclude file and where <HADOOP_CONF_DIR> is the directory for storing the Hadoop configuration files. For example, /etc/hadoop/conf.

 

First, ensure the DNS resolution is working or your /etc/hosts are updated and the passwordless connection is working with those hosts. Once the 10 new nodes are in the dfs.exclude file the namenode will consider them as bad nodes so no data will be replicated to them as long as these hosts remain in the dfs.exclude file once you have updated the NameNode with the new set of excluded DataNodes.

 

On the NameNode host machine, execute the following command:

 

su <HDFS_USER>
hdfs dfsadmin -refreshNodes

 

where <HDFS_USER> is the user owning the HDFS services

 

That should do the trick, once these hosts are visible in Ambari turn maintenance mode on so you don't receive any alerts

The day you will decide to add/enable these 10 new datanodes you will simply cp or mv the dfs.exclude to dfs.include file located <HADOOP_CONF_DIR>/dfs.include these nodes will start heartbeating and notifying the NameNode that  thes DataNodes are ready to start receiving files and participating in the data distribution in the cluster.

 

On the NameNode host machine remember to execute the following command:

 

su <HDFS_USER>
hdfs dfsadmin -refreshNodes

 

 

Don't forget to disable Maintenance mode on the new datanodes and remove them from  dfs.exclude file if you didn't rename or delete it.

 

Run the HDFS Balancer a tool for balancing the data across the storage devices of an HDFS cluster.

 

sudo -u hdfs hdfs balancer

 

The above balancer command has a couple of options either threshold or again the dfs.include and dfs.exclude see explanation below

 

                                              Include and Exclude Lists

 

When the include list is non-empty, only the datanodes specified in the list are balanced by the HDFS Balancer. An empty include list means including all the datanodes in the cluster. The default value is an empty list.

 

[-include [-f <hosts-file> | <comma-separated list of hosts>]]

 

The datanodes specified in the exclude list are excluded so that the HDFS Balancer does not balance those datanodes. An empty exclude list means that no datanodes are excluded. When a datanode is specified in both in the include list and the exclude list, the datanode is excluded. The default value is an empty list.

 

[-exclude [-f <hosts-file> | <comma-separated list of hosts>]]

 

 

If no dfs.include file is specified, all DataNodes are considered to be included in the cluster (unless excluded explicitly in the dfs.exclude file). The dfs.hosts and dfs.hosts.exclude properties in hdfs-site.xml are used to specify the dfs.include and dfs.exclude files.

Hope that helps 

Highlighted

Re: how to disable new datanode machine from data replication

little question

 

why just not stop the service - HDFS on each new data node  and set it to maintenance mode ?

Michael-Bronson
Don't have an account?
Coming from Hortonworks? Activate your account here