Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How-to: Manage Heterogeneous Hardware for Apache Hadoop using Cloudera Manager

Highlighted

How-to: Manage Heterogeneous Hardware for Apache Hadoop using Cloudera Manager

Master Collaborator

This new how-to contains important info about how to use role groups and host templates to easily expand an existing CDH cluster onto heterogeneous hardware:

 

http://blog.cloudera.com/blog/2013/07/managing-heterogeneous-hardware-for-apache-hadoop-using-cloude...

1 REPLY 1

Re: How-to: Manage Heterogeneous Hardware for Apache Hadoop using Cloudera Manager

Contributor

Thanks - that is helpful info.

 

First, has any of these changed now that we are in 5.8?  It seems the UI is different but the procedure remains the same, but want to verify.

 

Second, what about a more involved (and perhaps more realistic) scenario?  Say a production cluster running multiple services (e.g. Hive, Spark, HBase, Solr), now adding new nodes of different CPU, memory, and disk configs?  In particular, Cloudera Manager wizard calculate customized defaults for the many, many memory/resource allocation parameters across all the services based on available #cores and RAM.  We want to still keep this feature when we use Host Template.  Can we then specify the new role groups partially?

 

  1. Create role groups for the new nodes
  2. Change parameters in the new role groups where differences are clear, like dfs.data.dir and log locations
  3. Create host template from the new groups
  4. Add Hosts using the host template
  5. Cloudera Manager wizard automatically generate the YARN/memory/resource settings based on detected hardware, but presents them to the admin for acceptance.  If rejected, old values copied from the existing role groups will be used.

 

Thanks,

Miles

 

Don't have an account?
Coming from Hortonworks? Activate your account here