example blueprint for cloud break deploying on openstack using isilon

[#1]

Is it possible to use cloud break and ambari blueprints to deploy an EMC isilon cluster using openstack?

Can you post an example? For example, can you specify in an ambari blueprint the location of the name node and data node but have ambari skip the install?

[#2]

Hi,

Cloudbreak uses Heat templates to set up networking, launch VMs and attach disks to them. The volume management is abstracted under Cinder on OpenStack, therefore it is not relevant for Cloudbreak that what storage backend is used under the hood. Although we have never tested EMC Isilon, but if it can be managed through Cinder then Cloudbreak can handle it.

For your reference I copy here the Heat template fragment used by Cloudbreak, that is responsible to create the necessary cinder volume resources:

type: OS::Cinder::Volume
properties:
  name: hdfs-volume
  size: ${volume.size}

The complete template is here:

https://github.com/sequenceiq/cloudbreak/blob/master/cloud-openstack/src/main/resources/templates/op...

Attila

[#3]

@Attila Kanto

For iSilon, we need to add a node to the cluster using Ambari manual registration after the cluster is provisioned. So first we would need to create a blueprint with no data or name nodes, deploy it on open stack, and then after the cluster is created add the data and name node using manual registration.

It would probably be ok to have the manual registration as a manual step but would be even better if it could be automated.

[#4]

What do you do during manual registration? Do you install additional packages or just configure the data and namenode differently? In the latter case you can put those properties into the blueprint and you can do the installation and configuration in one step. If you are doing e.g. package installation and configuring things outside Hadoop config files the I see two options:

Basic Option (with manual steps):

• create a blueprint what is suitable for you needs just leave out the data a name node components. The blueprints are extremely flexible e.g for your reference I am attaching a very simple valid blueprint which contains only Zookeeper
• install the cluster and after installation just add services manually and configure Isilon

Advanced Option:

• You can write an Ambari Management Pack which contains the definition of EMC Isilon service: https://cwiki.apache.org/confluence/display/AMBARI/Management+Packs
• Install it with recipes: http://sequenceiq.com/cloudbreak-docs/latest/recipes/
• You can use that new service from blueprint

Attila

[#5]

@Attila Kanto This looks like the information I was looking for. Can you post this as an answer and I will accept it?

[#6 · Accepted solution]

Hi @cduby,

As requested:

What do you do during manual registration? Do you install additional packages or just configure the data and namenode differently? In the latter case you can put those properties into the blueprint and you can do the installation and configuration in one step. If you are doing e.g. package installation and configuring things outside Hadoop config files the I see two options:

Basic Option (with manual steps):

• create a blueprint what is suitable for you needs just leave out the data a name node components. The blueprints are extremely flexible e.g for your reference I am attaching a very simple valid blueprint which contains only Zookeeper
• install the cluster and after installation just add services manually and configure Isilon

Advanced Option:

• You can write an Ambari Management Pack which contains the definition of EMC Isilon service:https://cwiki.apache.org/confluence/display/AMBARI/Management+Packs
• Install it with recipes: http://sequenceiq.com/cloudbreak-docs/latest/recipes/
• You can use that new service from blueprint

Thanks,

Attila

[#7]

@cduby Sorry for posting to this closed issue. I'm interested to know if you were truly able to deploy this way though.

I have never done a Cloudbreak installation to Openstack with Isilon, so I'd love to find out more about what you did.