Skip to main content
yagoaparecidoti
Gold ⭐️
April 24, 2024
Solved

deploy rack concept in cluster

we are planning to deploy rack concept in cdh 6.3.4 cluster

the cluster is currently in the "/default" rack

i have some questions:

1. what would be the best scenario to deploy the rack in the cluster?
2. will this rack deployment cause unavailability in the cluster?
3. what are the “strengths” and “weaknesses” when deploying the rack in the cluster?

Asfahan
AsfahanAnswer
Gold ⭐️
May 16, 2024

so, a cluster with 3 racks, we will have 1 block for rack01, 1 block for rack02 and 1 block for rack03, right: Correct
but NN can also input  1 block on rack01 and 2 blocks for rack02 different DN, it works on both ways with 3 racks

8 replies

Asfahan
Gold ⭐️
Gold ⭐️
May 13, 2024

Hi Team

Below is your answer:

1. what would be the best scenario to deploy the rack in the cluster?

Ans. When you have huge read/writes going on DN and have huge no of DN then toreduce the network traffic while file read/write, which improves the cluster performance
2. will this rack deployment cause unavailability in the cluster?

This rack will contain equal no of DN suppose you have 9 DN with 3 rack then each rack will have 3 DN now block placement will be 1-2 Block in each rack with different DN so suppose one rack will fall then you will have another 2 rack to take care of read/write

3. what are the “strengths” and “weaknesses” when deploying the rack in the cluster?

Ans.

Strength: High Availabilty , Performance improvement

Weakness: not seen any with RF=3 blocks but if you have less than 3 then there might be scenario that 2 blocks will be on same rack but in different DN if that rack falls then it will cause unavailabilty

Translate
yagoaparecidoti
Gold ⭐️
May 13, 2024

hi @Asfahan 

we are planning to configure just 2 racks, would the scenario be the same?

example:

rack_01, with 10 DN hosts
rack_02, with 11 DN hosts

Translate
Asfahan
Gold ⭐️
Gold ⭐️
May 13, 2024

@yagoaparecidoti 

then one each rack will have 2 replica/each rack in different DN so if one rack goes down other will be available but its better to have 3 rack with 7 DN each as per your scenario

Translate
yagoaparecidoti
Gold ⭐️
May 13, 2024

@Asfahan 

when implementing the rack concept, isn't the distribution of blocks 1 block for rack_01 and 2 blocks for rack_02?

Translate
Asfahan
Gold ⭐️
Gold ⭐️
May 14, 2024

yes it will if you will my 2nd last comment i mention the same that 1 blocks will be in rack1 and 2 blocks will be in rack2 on different DN if you will have 3 racks then either 1 blocks each rack or same 1 block and 2 blocks in each rack


Translate
yagoaparecidoti
Gold ⭐️
May 14, 2024

@Asfahan 

hmm...ok!

so, a cluster with 2 racks, we will have 1 block for rack01 and 2 blocks for rack02, ok.

so, a cluster with 3 racks, we will have 1 block for rack01, 1 block for rack02 and 1 block for rack03, right?

Translate
Asfahan
Gold ⭐️
AsfahanAnswer
Gold ⭐️
May 16, 2024

so, a cluster with 3 racks, we will have 1 block for rack01, 1 block for rack02 and 1 block for rack03, right: Correct
but NN can also input  1 block on rack01 and 2 blocks for rack02 different DN, it works on both ways with 3 racks

Translate
yagoaparecidoti
Gold ⭐️
May 16, 2024

@Asfahan 

thank you very much for clarifying 😉

Translate