question Reduce program hosting location in Archives of Support Questions (Read Only)

Reduce program hosting location

ahamedfasil — Thu, 18 Aug 2016 13:10:24 GMT

Documentation says, map tasks are run in DataNodes and have data locality constraints which the scheduler tries to honor and reduce tasks can run anywhere in the cluster. The statement "can run anywhere in the cluster" for reduce tasks, is referring to only DataNodes in the cluster OR is ResourceManager machine also considered part of the cluster, so that Reduce tasks are allowed to run in ResourceManager also ??

Re: Reduce program hosting location

shiremath — Thu, 18 Aug 2016 13:21:42 GMT

reduce tasks "can run anywhere in the cluster" means on any of the node which has "Node Manager" installed on it

Re: Reduce program hosting location

KuldeepK — Thu, 18 Aug 2016 13:22:11 GMT

@Fasil Ahamed

"can run anywhere in the cluster" - Meaning anywhere where nodemanagers have been deployed, it can be any slave node or master node if nodemanager is installed on master nodes.

Hope this information helps!

Please do let me know if you have any further question.

Re: Reduce program hosting location

KuldeepK — Mon, 29 Aug 2016 08:31:03 GMT

@Fasil Ahamed - Can you please accept the appropriate answer?