OpenStack hardware architecture for running HDP

[#1]

Hello all,

I'm planning to have my own private cloud using OpenStack. I've already seen some sample architectures to do so, like the following one: http://docs.openstack.org/openstack-ops/content/example_architecture.html

My ideas is to have a regular private cloud but also capable to run HDP efficiently.

My question then is the next one: Which architecture should I use in order to run a HDP in my OpenStack cloud? There is any architecture sample out there to achieve that?

My guess is about having 4 compute nodes (with enough RAM, CPU and local storage, Cinder, to run many VMs as Hadoop nodes), 1 controller node (API services, high availability services, Neutron) and 1 Swift node (for all storage but Hadoop nodes storage).

What do you think about this possible architecture? Does it make sense?

Thanks!

[#2 · Accepted solution]

Yes you can use openstack for hadoop, because we are running successfully hdp on openstack since from last 9 months. And for hadoop node storage increase the size of cow2 image.

[#3]

Do you think having just 1 controller node is enough? This node will run Neutron, OpenStack components API services, OS and some monitoring staff as well.

[#4]

@Marc Cardus

Yes 1 controller is enough. Because you don't have big environment. I am running with same configuration 4 compute node 1 controller and I don't see any issues to monitor openstack components.

[#5]

For a size like this is fine - you can check these Ansible scripts in order to set up an OpenStack, than use Cloudbreak.

[#6]

@Marc Cardus

If you you agree with answer, then lets close this one.