Created 03-07-2017 10:34 AM
We are having a 6 node cluster ( 2 master and 3 slave nodes and 1 Edge Node) on AWS VM's deployed using Ambari. I need to replace the old VM's with VM's of increased size and memory.. What would be the best startegy t o replace the existing 3 slave nodes with a new set of slave nodes with a different instance type. Since we have some data in HDFS. what would be the best strategy to retain the data and bring up the cluster with new nodes.
Will it be advisable to replace Namenode or just Slave Nodes and Edge Node?
Created 03-07-2017 12:45 PM
you should add new nodes to the cluster and rebalance cluster, then decommission each node one at a time. Since you're replacing three nodes and that's all of your hdfs, it make take some time. Consider larger footprint as you know hdfs maintains replication factor of 3. You're best suited to use a tool like
Cloudbreak or if you're only running ETL, discovery, data science workloads, you can try Hortonworks Data Cloud. Both can add and remove instances as well as provision new instances with new machine type easily.
Created 03-07-2017 04:35 PM
Yes thats what my understanding is. Thanks for confirming.
By any chance is there a plan in future for cloudbreak to deploy cluster on already running VM's in cloud?
Created 03-07-2017 04:43 PM
not sure, I'm not privy to the Cloudbreak roadmap. Perhaps you'd want to open this as a new HCC question and someone from the Cloudbreak team can respond?
Created 03-08-2017 05:44 AM
Ok sure. Just wanted to know about Namenode as well in case that also needs an upgrade to VM of higher memory and CPU. What should be the ideal scenario in that case?
Thanks
Created 03-08-2017 08:48 AM
Add two new nodes, then in Ambari there's an option to move namenode in the HDFS section.
Created 09-05-2018 12:05 PM
> Both can add and remove instances as well as provision new instances with new machine type easily.
Could you please point where that option could be located in the UI or CLI of Cloudbreak? Thank you!