Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

What is the releatiohship between yarn container and Block size in hdfs?

avatar
Rising Star

What is the releatiohship between yarn container and Block size in hdfs

1 ACCEPTED SOLUTION

avatar
Super Guru

@ANSARI FAHEEM AHMEDyarn container is a collection of physical resources like CPU,memory and disk.while HDFS block size is chunk on file system where actual read and write happen.

View solution in original post

3 REPLIES 3

avatar
Super Guru

@ANSARI FAHEEM AHMEDyarn container is a collection of physical resources like CPU,memory and disk.while HDFS block size is chunk on file system where actual read and write happen.

avatar
Rising Star

@Rajkumar Singh: Yes, means no relation.

avatar
Master Guru

I mean a bit relation is there. Normally MapReduce will create one Map task for every block. ( Unless small split merge is switched on ). And one map task will run in one container. So half the block size means twice the number of containers running. ( Again not always true since Pig/Tez merge small blocks together using something called the CombineFileInputFormat)