As per support matrix the combination of RHEL8.9 with CM -7.10.1 is not supported thats why you are facing such issue, We don't have any CU with same setup as we always recommend in sanity check to go through support matrix if its says supported then only proceed
... View more
Hello , can you please share the code of distcp you were using with all necessary parameter ALso did you try to run the same in Debug console to understand more HADOOP_ROOT_LOGGER=DEBUG,console
... View more
so, a cluster with 3 racks, we will have 1 block for rack01, 1 block for rack02 and 1 block for rack03, right: Correct but NN can also input 1 block on rack01 and 2 blocks for rack02 different DN, it works on both ways with 3 racks
... View more
yes it will if you will my 2nd last comment i mention the same that 1 blocks will be in rack1 and 2 blocks will be in rack2 on different DN if you will have 3 racks then either 1 blocks each rack or same 1 block and 2 blocks in each rack
... View more
@yagoaparecidoti then one each rack will have 2 replica/each rack in different DN so if one rack goes down other will be available but its better to have 3 rack with 7 DN each as per your scenario
... View more
Hi Team Below is your answer: 1. what would be the best scenario to deploy the rack in the cluster? Ans. When you have huge read/writes going on DN and have huge no of DN then toreduce the network traffic while file read/write, which improves the cluster performance 2. will this rack deployment cause unavailability in the cluster? This rack will contain equal no of DN suppose you have 9 DN with 3 rack then each rack will have 3 DN now block placement will be 1-2 Block in each rack with different DN so suppose one rack will fall then you will have another 2 rack to take care of read/write 3. what are the “strengths” and “weaknesses” when deploying the rack in the cluster? Ans. Strength: High Availabilty , Performance improvement Weakness: not seen any with RF=3 blocks but if you have less than 3 then there might be scenario that 2 blocks will be on same rack but in different DN if that rack falls then it will cause unavailabilty
... View more
Hi Team, Did you use hdfs dfsadmin -report command to check the dfs and non-dfs used, dfs is used by HDFS so i blieve 9TB is complete storage with dfs and nondfs. Now when you copying the file to hdfs have you enable the debug HADOOP_ROOT_LOGGER=DEBUG,console Also try to run the below command it will copy the file from local to hdfs hdfs dfs -Ddfs.replication=#noofdatanodes -put /tmp/<3mbfile> /tmp
... View more
Hi Team, looks like you have 3 DN with rack topology have you check how many racks there as per rack awareness 1 block will be on 1 DN in each rach or max 2 block in each rack with different datanode You can try triggering the block report manually from DN to NN hdfs dfsadmin -triggerBlockReport <datanode>:<ipc_port>
... View more
Hello Olivier, if the issue resolved please mark one of above thread as solution if not then provide us the update so that we can prepare next action plan for you. Regards, Asfahan Zafar
... View more
