question Re: OpenStack hardware architecture for running HDP in Archives of Support Questions (Read Only)

OpenStack hardware architecture for running HDP

marc_cardus — Wed, 08 Jun 2016 01:05:12 GMT

Hello all,

I'm planning to have my own private cloud using OpenStack. I've already seen some sample architectures to do so, like the following one: http://docs.openstack.org/openstack-ops/content/example_architecture.html

My ideas is to have a regular private cloud but also capable to run HDP efficiently.

My question then is the next one: Which architecture should I use in order to run a HDP in my OpenStack cloud? There is any architecture sample out there to achieve that?

My guess is about having 4 compute nodes (with enough RAM, CPU and local storage, Cinder, to run many VMs as Hadoop nodes), 1 controller node (API services, high availability services, Neutron) and 1 Swift node (for all storage but Hadoop nodes storage).

What do you think about this possible architecture? Does it make sense?

Thanks!

Re: OpenStack hardware architecture for running HDP

ashneesharma88 — Thu, 09 Jun 2016 13:38:02 GMT

Yes you can use openstack for hadoop, because we are running successfully hdp on openstack since from last 9 months. And for hadoop node storage increase the size of cow2 image.

Re: OpenStack hardware architecture for running HDP

marc_cardus — Thu, 09 Jun 2016 15:54:47 GMT

Do you think having just 1 controller node is enough? This node will run Neutron, OpenStack components API services, OS and some monitoring staff as well.

Re: OpenStack hardware architecture for running HDP

jmatyas — Fri, 10 Jun 2016 23:26:26 GMT

For a size like this is fine - you can check these Ansible scripts in order to set up an OpenStack, than use Cloudbreak.

Re: OpenStack hardware architecture for running HDP

ashneesharma88 — Mon, 13 Jun 2016 14:50:59 GMT

@Marc Cardus

Yes 1 controller is enough. Because you don't have big environment. I am running with same configuration 4 compute node 1 controller and I don't see any issues to monitor openstack components.

Re: OpenStack hardware architecture for running HDP

ashneesharma88 — Wed, 15 Jun 2016 22:40:01 GMT

@Marc Cardus

If you you agree with answer, then lets close this one.