Created 06-30-2016 09:19 AM
What is the releatiohship between yarn container and Block size in hdfs
Created 06-30-2016 09:30 AM
@ANSARI FAHEEM AHMEDyarn container is a collection of physical resources like CPU,memory and disk.while HDFS block size is chunk on file system where actual read and write happen.
Created 06-30-2016 09:30 AM
@ANSARI FAHEEM AHMEDyarn container is a collection of physical resources like CPU,memory and disk.while HDFS block size is chunk on file system where actual read and write happen.
Created 06-30-2016 09:38 AM
@Rajkumar Singh: Yes, means no relation.
Created 06-30-2016 10:36 AM
I mean a bit relation is there. Normally MapReduce will create one Map task for every block. ( Unless small split merge is switched on ). And one map task will run in one container. So half the block size means twice the number of containers running. ( Again not always true since Pig/Tez merge small blocks together using something called the CombineFileInputFormat)