Reply
New Contributor
Posts: 3
Registered: ‎01-24-2017

Reorganisation of existing CHD 5 Cluster

Hello All,

I am quite new to CDH.

I have come across a CDH v5.4.8 Cluster, for a POC setup consisting of 6 Nodes. 1 Name Node & 5 Data Nodes.

I know 6 nodes are not enough, but what will be best role assignment and distribution for the given cluster for POC.

Services used will be will be Spark , Solr & HBase.
Also how can i test configure & test HA for the same services ?

Secondly , Requirement is for making Cloudera Manager HA.

I have gone through documentation of Cloudera for the same.

It says that,
1. Embedded PostgreSQL is not supported.
2. Cloudera Manager Server & Services should not be on same host.

I have all of it installed on same Name Node. How can i move it from the existing Name Node to a new server ?

If any details required let me know.

I know above scenario is a bit mess but challange herr is to reorganisation of cluster.

Thanks,
Dark
Posts: 642
Topics: 3
Kudos: 105
Solutions: 67
Registered: ‎08-16-2016

Re: Reorganisation of existing CHD 5 Cluster

Services used will be will be Spark , Solr & HBase.

Each of these are its own resource manager/management and scheduler, unless you are using Spark in YARN. So keep that in mind as you will need to divide up the cluster resources to each.

Also how can i test configure & test HA for the same services ?

CM provides wizards for both NN and RM HA. I would read through the docs though in case something goes wrong so that you will know what didn't get finished. Testing is usually straight forward, take the active down and try to use the service. Other services have HA as well, some more complicated that others; testing is the same. Read the Cloudera docs or open specific questions about HA for a given service.

Secondly , Requirement is for making Cloudera Manager HA.

I have yet to use or need CM HA. It is the command and control center for the cluster but the cluster will continue to function if it goes down for a time. Yes on both requirements as both CM hosts will need to be able to reach the DB. I am not sure on the intent of the second point. The Service and and Host services write data to a local tsdb on disk. That can be IO intensive so there is some benefit to moving it elsewhere or just sizing a single node correctly.

How can i move it from the existing Name Node to a new server ?

I haven't done this in a while. It would be easier with NN HA in place already. The key is that you need to keep the NN metadata intact. Rough steps are to stop the HDFS, then take a backup of the NN metadata (check your configs for the location), then restore it to the new machine to the same location, ensure that permissions and ownership are identical, then remove the service from the old host and add it to the new, restart and push new configs.

Note: CM 5.8.1 has at Migrate Roles option on the instance page of the HDFS service. I have not tried this feature but it probably takes much of the hassle mentioned above off your hands.
Announcements