I thought decommissioning all of them would automatically copy data to commissioned nodes. That did not happen though. I currently have 4 DN's in the cluster. I need to remove all 4 of them add 3 new ones instead. I need to maintain the data ofcourse.
What's the way forward?
This may not be perfect answer for you but still you can try this if you want:
stop one of the datanode in cluster and add new data node. rebalance the blocks by using hdfs rebalancer. decommission the stopped data node in cluster. repeat the same process until all data nodes will be added by cluster.