question Re: deploy rack concept in cluster in Support Questions

deploy rack concept in cluster

yagoaparecidoti — Wed, 24 Apr 2024 18:32:44 GMT

we are planning to deploy rack concept in cdh 6.3.4 cluster

the cluster is currently in the "/default" rack

i have some questions:

1. what would be the best scenario to deploy the rack in the cluster?
2. will this rack deployment cause unavailability in the cluster?
3. what are the “strengths” and “weaknesses” when deploying the rack in the cluster?

Re: deploy rack concept in cluster

Asfahan — Mon, 13 May 2024 10:06:10 GMT

Hi Team

Below is your answer:

1. what would be the best scenario to deploy the rack in the cluster?

Ans. When you have huge read/writes going on DN and have huge no of DN then toreduce the network traffic while file read/write, which improves the cluster performance
2. will this rack deployment cause unavailability in the cluster?

This rack will contain equal no of DN suppose you have 9 DN with 3 rack then each rack will have 3 DN now block placement will be 1-2 Block in each rack with different DN so suppose one rack will fall then you will have another 2 rack to take care of read/write

3. what are the “strengths” and “weaknesses” when deploying the rack in the cluster?

Ans.

Strength: High Availabilty , Performance improvement

Weakness: not seen any with RF=3 blocks but if you have less than 3 then there might be scenario that 2 blocks will be on same rack but in different DN if that rack falls then it will cause unavailabilty

Re: deploy rack concept in cluster

yagoaparecidoti — Mon, 13 May 2024 12:46:39 GMT

hi @Asfahan

we are planning to configure just 2 racks, would the scenario be the same?

example:

rack_01, with 10 DN hosts
rack_02, with 11 DN hosts

Re: deploy rack concept in cluster

Asfahan — Mon, 13 May 2024 15:21:21 GMT

@yagoaparecidoti

then one each rack will have 2 replica/each rack in different DN so if one rack goes down other will be available but its better to have 3 rack with 7 DN each as per your scenario

Re: deploy rack concept in cluster

yagoaparecidoti — Mon, 13 May 2024 17:15:15 GMT

@Asfahan

when implementing the rack concept, isn't the distribution of blocks 1 block for rack_01 and 2 blocks for rack_02?

Re: deploy rack concept in cluster

Asfahan — Tue, 14 May 2024 07:09:32 GMT

yes it will if you will my 2nd last comment i mention the same that 1 blocks will be in rack1 and 2 blocks will be in rack2 on different DN if you will have 3 racks then either 1 blocks each rack or same 1 block and 2 blocks in each rack

Re: deploy rack concept in cluster

yagoaparecidoti — Tue, 14 May 2024 12:56:46 GMT

@Asfahan

hmm...ok!

so, a cluster with 2 racks, we will have 1 block for rack01 and 2 blocks for rack02, ok.

so, a cluster with 3 racks, we will have 1 block for rack01, 1 block for rack02 and 1 block for rack03, right?

Re: deploy rack concept in cluster

Asfahan — Thu, 16 May 2024 07:20:21 GMT

so, a cluster with 3 racks, we will have 1 block for rack01, 1 block for rack02 and 1 block for rack03, right: Correct
but NN can also input 1 block on rack01 and 2 blocks for rack02 different DN, it works on both ways with 3 racks

Re: deploy rack concept in cluster

yagoaparecidoti — Thu, 16 May 2024 12:33:22 GMT

@Asfahan

thank you very much for clarifying 😉