Reply
Explorer
Posts: 13
Registered: ‎07-08-2015

Hadoop daemons on physical machines

Does namenode and jobtracker run on the same physical machine or separate machines? Is there a tasktracker present in every datanode?

Cloudera Employee
Posts: 13
Registered: ‎08-19-2015

Re: Hadoop daemons on physical machines

[ Edited ]

In MapReduce V1 (non-YARN) and on a small cluster the JobTracker is typically run on the same hosts at the NameNode and you run TaskTracker on all the workers. In larger enviroments the JobTracker would be run elsewhere, not the NameNode or the DataNodes, depending on what you have avaliable.

 

For MapReduce V2(YARN) JobTracker has morphed into the ResourceManager and Job History Server and the TaskTracker is now the NodeManager

Well it all depends on how you set it up. Most of the services in CDH are designed to be run independently of one another allowing you to separate resource intensive services from one another. 

There is a great blog post:

 

How-to: Deploy Apache Hadoop Clusters Like a Boss

 

That goes into detail on best practices on deployment. I think the best thing on it is this picutre:

 

Services-architecture.png

 

Thanks

Josh