datanode config groups?

[#1]

Hi Expert,

Currently, our cluster's hosts has 12 data disks(900G) installed besides the OS disk. and I created files systems /data01~data12 for every data disk. and set dfs.datanode.data.dir

/data01/hadoop/hdfs/data,/data02/hadoop/hdfs/data,/data03/hadoop/hdfs/data,/data04/hadoop/hdfs/data,/data05/hadoop/hdfs/data,/data06/hadoop/hdfs/data,/data07/hadoop/hdfs/data,/data08/hadoop/hdfs/data,/data09/hadoop/hdfs/data,/data10/hadoop/hdfs/data,/data11/hadoop/hdfs/data,/data12/hadoop/hdfs/data

Now, I'd like add several hosts with only 5 data disks(1.8T) installed to the cluster as datanodes. can I do it by creating 5 files systems /data01~data05 and adding new config groups with the dfs.datanode.data.dir in Ambari?

/data01/hadoop/hdfs/data,/data02/hadoop/hdfs/data,/data03/hadoop/hdfs/data,/data04/hadoop/hdfs/data,/data05/hadoop/hdfs/data

For such exending capablity scenario, What is the best practice?

[#2]

@Huahua Wei

I guess you should use the Config Group concept via Ambari to achieve the same. You can add those new hosts to your cluster using ambari and then create a new HDFS config group and choose those hosts where you want different value for the mentioned property.

http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-user-guide/content/using_host_conf...

[#3]

This url seems to manage the existing hosts, how about adding new hosts?

[#4]

@Huahua Wei

You can add new hosts to ambari cluster via Ambari itself as mentioned in the following link.

http://hortonworks.com/hadoop-tutorial/using-apache-ambari-add-new-nodes-existing-cluster/

- Once those hosts are added to your cluster then you can deploy your desired services/components to them like DataNode via ambari. The create the config groups.