Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

with HDFS and Yarn with version 2.7.1.2.5 , do we still need MapReduce2

avatar
Contributor

In our Stack we installed with HDFS and Yarn with version 2.7.1.2.5.

Do we still need to install MapReduce2 ( which as two components like History Server and MapReduce2 Clients) . If so, On what nodes do we need to install MapReduce2 Clients ( like only on Data nodes or yarn hosts or MapReduce2 host only)

Regards

JJ

1 ACCEPTED SOLUTION

avatar
Master Mentor

@Jacqualin jasmin

The simple answer is YES for MR

This whole depends on the number of nodes in your cluster and the desired setup HA or NOT. Usually, there are Masternodes ,Datanodes and edge node (Clients Only)

Masternode (if HA the recommended setup at least 3 nodes)

- NAMENODE. 
- JOURNALNODE. 
- ZKFC. 
- NODEMANAGER. 
- HBASE_MASTER(Optional). 
- HISTORYSERVER. 
- APP_TIMELINE_SERVER. 
- RESOURCEMANAGER. 
- ZOOKEEPER_SERVER.(at least 2 ) 
- JOURNALNODE. 
- MYSQL_SERVER. 
- ZKFC. 
- HIVE_SERVER. 
- HIVE_METASTORE. 
- WEBHCAT_SERVER. 
- ZOOKEEPER_CLIENT. 

Datanode (As desired)

- JOURNALNODE. 
- NODEMANAGER. 
- HBASE_REGIONSERVER(if Hbase installed). 
- ZOOKEEPER_CLIENT. 
- DATANODE 

Edgenode ( At least one)

- YARN_CLIENT. 
- ZOOKEEPER_CLIENT. 
- RANGER_ADMIN. 
- RANGER_USERSYNC. 
- METRICS_MONITOR. 
- METRICS_COLLECTOR. 
- ZEPPELIN_MASTER. 
- INFRA_SOLR. 
- INFRA_SOLR_CLIENT. 
- HBASE_CLIENT. 
- HDFS_CLIENT. 
- TEZ_CLIENT. 
- YARN_CLIENT. 
- HIVE_CLIENT. 
- MAPREDUCE2_CLIENT

In the above setup, some components are optional. See official Hortonworks document

View solution in original post

4 REPLIES 4

avatar
Guru

Hi @Jacqualin jasmin

Yes you should still install MapReduce for the history server, and the client tools. Certain components still use MapReduce under the hood, and having it installed does not usually cause harm or take up too many resources. The history server can be installed on any master node you like and makes sense to do so, while the clients are typically installed on edge (or data) nodes where users will have access.

avatar
Contributor

Thank Sonu, it helps me alot . keep up this spirit

avatar
Master Mentor

@Jacqualin jasmin

The simple answer is YES for MR

This whole depends on the number of nodes in your cluster and the desired setup HA or NOT. Usually, there are Masternodes ,Datanodes and edge node (Clients Only)

Masternode (if HA the recommended setup at least 3 nodes)

- NAMENODE. 
- JOURNALNODE. 
- ZKFC. 
- NODEMANAGER. 
- HBASE_MASTER(Optional). 
- HISTORYSERVER. 
- APP_TIMELINE_SERVER. 
- RESOURCEMANAGER. 
- ZOOKEEPER_SERVER.(at least 2 ) 
- JOURNALNODE. 
- MYSQL_SERVER. 
- ZKFC. 
- HIVE_SERVER. 
- HIVE_METASTORE. 
- WEBHCAT_SERVER. 
- ZOOKEEPER_CLIENT. 

Datanode (As desired)

- JOURNALNODE. 
- NODEMANAGER. 
- HBASE_REGIONSERVER(if Hbase installed). 
- ZOOKEEPER_CLIENT. 
- DATANODE 

Edgenode ( At least one)

- YARN_CLIENT. 
- ZOOKEEPER_CLIENT. 
- RANGER_ADMIN. 
- RANGER_USERSYNC. 
- METRICS_MONITOR. 
- METRICS_COLLECTOR. 
- ZEPPELIN_MASTER. 
- INFRA_SOLR. 
- INFRA_SOLR_CLIENT. 
- HBASE_CLIENT. 
- HDFS_CLIENT. 
- TEZ_CLIENT. 
- YARN_CLIENT. 
- HIVE_CLIENT. 
- MAPREDUCE2_CLIENT

In the above setup, some components are optional. See official Hortonworks document

avatar
Contributor

Thank you very much Geoffrey, for your insights and supporting the community.