Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

what are steps of easiest way to create an HDP cluster on raw HP servers

what are steps of easiest way to create an HDP cluster on raw HP servers

New Contributor

Hi,

I am new in making private clouds and hadoop ecosystem. We have three HP servers without ESX or openstack, and want to create a private cloud and then have an HDP cluster with Ambari. It would be appreciated, If somebody could explain the steps of creating a cloud and hadoop cluster by using (ESXi, openstack, cloudbreak, Ambari and ....). there are some tutorials to do this, but they are confusing to me. some of our requirements are as follows:

1- we must have at least 100TB HDFS capacity.

2- we should be able to add new nodes for increasing the capacity of HDFS and processing power.

3- we should be able to share the HDP cluster with multiple users. each user should be able to run own map/reduce job.

Sincerely,

4 REPLIES 4

Re: what are steps of easiest way to create an HDP cluster on raw HP servers

@Ali Mohammadi Shanghoshabad

You have this option which covers all the 3 requirements:

  1. Install and setup an OpenStack version compatible with Cloudbreak
  2. Install Cloudbreak Deployer and start Cloudbreak
  3. Create an OpenStack credential (which connects your OpenStack account with Cloudbreak) and infrastructure templates in Cloudbreak
  4. Create or reuse a blueprints matching your HDP workload type
  5. Launch cluster(s)
  6. Configure autoscale and add users via Ambari as needed

Hope this helps!

Re: what are steps of easiest way to create an HDP cluster on raw HP servers

New Contributor

@pdarvasi

Thanks for your valuable and useful suggestion. Actually, my major problem is step one.

I need to have openstack on current hp servers and also need to extend it whenever it is necessary. shall I install ESXi and then create a VM and after that install openstack? or I can install openstack on raw servers and then can create a extendable private cloud without ESXi. In fact, I am confusing with creating clusters. For example, I thought, I have to manage a cluster of servers on Vsphere, a cluster of nodes on openstack, and then a hadoop cluster on Ambari. It's still confusing for me. It would be appreciated if you could give me more information or introduce some tutorials.

thanks a million,

ALi

Re: what are steps of easiest way to create an HDP cluster on raw HP servers

@Ali Mohammadi Shanghoshabad

That first step has very little to do with Hortonworks, that's why I was not elaborating on it in detail.

As a rule of thumb, if you can go with KVM hypervisor, you should go with it.

The reasons are:

  1. It os the only officially supported hypervisor for OpenStack, most OpenStack development is done with the KVM hypervisor
  2. Cloudbreak supports KVM for OpenStack.
  3. The reference architecture contains KVM as well.

Hope this helps!

Re: what are steps of easiest way to create an HDP cluster on raw HP servers

New Contributor

@pdarvas

thanks, I am going to check it.