if the scheduler is FIFO how the size of container is decide by AM ?

[#1]

in FIFO scheduler only one job is allowed to run at a time. so the resource of a node will further divides in container or not ? if yes then what is advantage of it as no other job is going to execute at same time?

[#2 · Accepted solution]

FIFO was the default scheduler in Hadoop1, when you deploy Apache Hadoop vanilla version. It is not used in production, as only 1 job can run. But 1 job can have many containers and each node in the cluster can have few of them running.

Size of the container has nothing to do with how many jobs are running on a cluster. It is decided by the map/reduce memory which asks for a container from YARN, which should be a multiple of yarn minimum allocation.

There are lot of details here, but keeping it simple the size of container is not related to the number of containers. Yes, for mathematical calculations and to find how many containers can run on a node, we say that the number is equal to total memory avail for yarn/container memory.

Size of a container is decided based on the request, whether it is map or reduce or spark task container etc.

[#3]

Thank you.!