Turning on Maintanance Mode on a data node

[#1]

Hi All,

I am working on an issue where in some misconfiguration has been done on our cluster. One of the datanode folder has been mounted on a root partition, rather than on a dedicated harddisk. Because of this root partition is getting filled.

While our Linux Admin team can unmount it from root partition and mount datanode folder on dedicated harddisk what will be the consequences of this on our cluster?

I have done background work and have figured out that I have to Turn on the maintanace mode on that datanode till the activity of mounting is done. Apart from this do I have to take care of anything prior to putting data node into Maintanace.

Do I have to turn on All services running on that datanode prior to putting data node into maintanace?

Request you guys input?

[#2 · Accepted solution]

Hi,

When you are trying to unmount or mount new data disk into datanode, node will be restarted ! So it is best to stop all the services on this node and to keep in maintenance mode !

PS: Take backup of your data partition to avoid data loss or corruption !

Cheers,

Ram

[#3]

you can try this

1) Turn on Maintenance at HDFS level and data node level

2) decommission the data node

3) Stop all the service on data node

4) Mount data disk

5) add data disk in HDFS configuration

6) add data node to the cluster

7) re-start all the effected services.

I would suggest you to create support ticket if you are having support license.

Regards,

Divakar

[#4]

You may also see UNDERREPLICATED BLOCK alerts after mounting to the right disk. This alert will disappear after some time once HDFS is caught up with the replication.

Thanks

Kumar