rack awareness configuration on cloudera 6.3.x cluster

Question

we are planning to separate cluster hosts into 2 physical racks, 10 hosts for each rack.

our doubt is:

as we only have two physical racks, is it still recommended to use the "rack" option in cloudera manager to separate the hosts?

rki_ · Accepted Answer

hi @yagoaparecidoti Yes, the Block for the file would be present on the Datanode in either of the rack. So if you write a file with RF 3, 1 block will be present on a rack and its 2 replicas will be present on different Datanodes of 2nd rack.

cjervis · Answer

Hi @yagoaparecidoti.

I'm not an expert but I checked documentation and found this:
Specifying Racks for Hosts

If I reading correctly you will have a total of twenty hosts across two racks so this statement would apply.

Cloudera Manager includes internal rack awareness scripts, but you must specify the racks where the hosts in your cluster are located. If your cluster contains more than 10 hosts, Cloudera recommends that you specify the rack for each host. HDFS, MapReduce, and YARN will automatically use the racks you specify.

I hope this helps.

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.