Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

What is the releatiohship between yarn container and Block size in hdfs?

Contributor

What is the releatiohship between yarn container and Block size in hdfs

1 ACCEPTED SOLUTION

@ANSARI FAHEEM AHMEDyarn container is a collection of physical resources like CPU,memory and disk.while HDFS block size is chunk on file system where actual read and write happen.

View solution in original post

3 REPLIES 3

@ANSARI FAHEEM AHMEDyarn container is a collection of physical resources like CPU,memory and disk.while HDFS block size is chunk on file system where actual read and write happen.

Contributor

@Rajkumar Singh: Yes, means no relation.

I mean a bit relation is there. Normally MapReduce will create one Map task for every block. ( Unless small split merge is switched on ). And one map task will run in one container. So half the block size means twice the number of containers running. ( Again not always true since Pig/Tez merge small blocks together using something called the CombineFileInputFormat)

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.