with HDFS and Yarn with version 2.7.1.2.5 , do we still need MapReduce2

[#1]

In our Stack we installed with HDFS and Yarn with version 2.7.1.2.5.

Do we still need to install MapReduce2 ( which as two components like History Server and MapReduce2 Clients) . If so, On what nodes do we need to install MapReduce2 Clients ( like only on Data nodes or yarn hosts or MapReduce2 host only)

Regards

JJ

[#2]

Hi @Jacqualin jasmin

Yes you should still install MapReduce for the history server, and the client tools. Certain components still use MapReduce under the hood, and having it installed does not usually cause harm or take up too many resources. The history server can be installed on any master node you like and makes sense to do so, while the clients are typically installed on edge (or data) nodes where users will have access.

[#3]

Thank Sonu, it helps me alot . keep up this spirit

[#4 · Accepted solution]

@Jacqualin jasmin

The simple answer is YES for MR

This whole depends on the number of nodes in your cluster and the desired setup HA or NOT. Usually, there are Masternodes ,Datanodes and edge node (Clients Only)

Masternode (if HA the recommended setup at least 3 nodes)

- NAMENODE. 
- JOURNALNODE. 
- ZKFC. 
- NODEMANAGER. 
- HBASE_MASTER(Optional). 
- HISTORYSERVER. 
- APP_TIMELINE_SERVER. 
- RESOURCEMANAGER. 
- ZOOKEEPER_SERVER.(at least 2 ) 
- JOURNALNODE. 
- MYSQL_SERVER. 
- ZKFC. 
- HIVE_SERVER. 
- HIVE_METASTORE. 
- WEBHCAT_SERVER. 
- ZOOKEEPER_CLIENT. 

Datanode (As desired)

- JOURNALNODE. 
- NODEMANAGER. 
- HBASE_REGIONSERVER(if Hbase installed). 
- ZOOKEEPER_CLIENT. 
- DATANODE 

Edgenode ( At least one)

- YARN_CLIENT. 
- ZOOKEEPER_CLIENT. 
- RANGER_ADMIN. 
- RANGER_USERSYNC. 
- METRICS_MONITOR. 
- METRICS_COLLECTOR. 
- ZEPPELIN_MASTER. 
- INFRA_SOLR. 
- INFRA_SOLR_CLIENT. 
- HBASE_CLIENT. 
- HDFS_CLIENT. 
- TEZ_CLIENT. 
- YARN_CLIENT. 
- HIVE_CLIENT. 
- MAPREDUCE2_CLIENT

In the above setup, some components are optional. See official Hortonworks document

[#5]

Thank you very much Geoffrey, for your insights and supporting the community.