Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

with HDFS and Yarn with version 2.7.1.2.5 , do we still need MapReduce2

Solved Go to solution

with HDFS and Yarn with version 2.7.1.2.5 , do we still need MapReduce2

New Contributor

In our Stack we installed with HDFS and Yarn with version 2.7.1.2.5.

Do we still need to install MapReduce2 ( which as two components like History Server and MapReduce2 Clients) . If so, On what nodes do we need to install MapReduce2 Clients ( like only on Data nodes or yarn hosts or MapReduce2 host only)

Regards

JJ

1 ACCEPTED SOLUTION

Accepted Solutions

Re: with HDFS and Yarn with version 2.7.1.2.5 , do we still need MapReduce2

Mentor

@Jacqualin jasmin

The simple answer is YES for MR

This whole depends on the number of nodes in your cluster and the desired setup HA or NOT. Usually, there are Masternodes ,Datanodes and edge node (Clients Only)

Masternode (if HA the recommended setup at least 3 nodes)

- NAMENODE. 
- JOURNALNODE. 
- ZKFC. 
- NODEMANAGER. 
- HBASE_MASTER(Optional). 
- HISTORYSERVER. 
- APP_TIMELINE_SERVER. 
- RESOURCEMANAGER. 
- ZOOKEEPER_SERVER.(at least 2 ) 
- JOURNALNODE. 
- MYSQL_SERVER. 
- ZKFC. 
- HIVE_SERVER. 
- HIVE_METASTORE. 
- WEBHCAT_SERVER. 
- ZOOKEEPER_CLIENT. 

Datanode (As desired)

- JOURNALNODE. 
- NODEMANAGER. 
- HBASE_REGIONSERVER(if Hbase installed). 
- ZOOKEEPER_CLIENT. 
- DATANODE 

Edgenode ( At least one)

- YARN_CLIENT. 
- ZOOKEEPER_CLIENT. 
- RANGER_ADMIN. 
- RANGER_USERSYNC. 
- METRICS_MONITOR. 
- METRICS_COLLECTOR. 
- ZEPPELIN_MASTER. 
- INFRA_SOLR. 
- INFRA_SOLR_CLIENT. 
- HBASE_CLIENT. 
- HDFS_CLIENT. 
- TEZ_CLIENT. 
- YARN_CLIENT. 
- HIVE_CLIENT. 
- MAPREDUCE2_CLIENT

In the above setup, some components are optional. See official Hortonworks document

4 REPLIES 4

Re: with HDFS and Yarn with version 2.7.1.2.5 , do we still need MapReduce2

Guru

Hi @Jacqualin jasmin

Yes you should still install MapReduce for the history server, and the client tools. Certain components still use MapReduce under the hood, and having it installed does not usually cause harm or take up too many resources. The history server can be installed on any master node you like and makes sense to do so, while the clients are typically installed on edge (or data) nodes where users will have access.

Re: with HDFS and Yarn with version 2.7.1.2.5 , do we still need MapReduce2

New Contributor

Thank Sonu, it helps me alot . keep up this spirit

Re: with HDFS and Yarn with version 2.7.1.2.5 , do we still need MapReduce2

Mentor

@Jacqualin jasmin

The simple answer is YES for MR

This whole depends on the number of nodes in your cluster and the desired setup HA or NOT. Usually, there are Masternodes ,Datanodes and edge node (Clients Only)

Masternode (if HA the recommended setup at least 3 nodes)

- NAMENODE. 
- JOURNALNODE. 
- ZKFC. 
- NODEMANAGER. 
- HBASE_MASTER(Optional). 
- HISTORYSERVER. 
- APP_TIMELINE_SERVER. 
- RESOURCEMANAGER. 
- ZOOKEEPER_SERVER.(at least 2 ) 
- JOURNALNODE. 
- MYSQL_SERVER. 
- ZKFC. 
- HIVE_SERVER. 
- HIVE_METASTORE. 
- WEBHCAT_SERVER. 
- ZOOKEEPER_CLIENT. 

Datanode (As desired)

- JOURNALNODE. 
- NODEMANAGER. 
- HBASE_REGIONSERVER(if Hbase installed). 
- ZOOKEEPER_CLIENT. 
- DATANODE 

Edgenode ( At least one)

- YARN_CLIENT. 
- ZOOKEEPER_CLIENT. 
- RANGER_ADMIN. 
- RANGER_USERSYNC. 
- METRICS_MONITOR. 
- METRICS_COLLECTOR. 
- ZEPPELIN_MASTER. 
- INFRA_SOLR. 
- INFRA_SOLR_CLIENT. 
- HBASE_CLIENT. 
- HDFS_CLIENT. 
- TEZ_CLIENT. 
- YARN_CLIENT. 
- HIVE_CLIENT. 
- MAPREDUCE2_CLIENT

In the above setup, some components are optional. See official Hortonworks document

Highlighted

Re: with HDFS and Yarn with version 2.7.1.2.5 , do we still need MapReduce2

New Contributor

Thank you very much Geoffrey, for your insights and supporting the community.

Don't have an account?
Coming from Hortonworks? Activate your account here