Support Questions
Find answers, ask questions, and share your expertise

documentation for blueprint processor

Expert Contributor

The cluster creation template suggests here that host_groups consists of a name with a set of hosts.

Most of the ambari blueprint examples create multiple host_groups in master slave paradigm and then have only one host per host_group.

How do we access a particular host within a host_group ? Can you point me to the documentation for the blueprint processor which injects and processes these values ?

{
  "blueprint" : "mycluster",
  "host_groups" :[
    {
      "name" : "zk",
      "hosts" : [
        {
                      "fqdn" : "my.host1.com" ,          
                      "fqdn" : "my.host2.com" ,          
                      "fqdn" : "my.host3.com"           
                  }
      ]
    }
  ]
}


Then in the blueprints we can access using %HOSTGROUP::zk% .. how do i access the second host for instance ?

%HOSTGROUP::zk[1]% ?

Rather than defining machines which host the roles I am attempting to group the roles into the hostgroups and then so depending on my envronment the right machines get the right roles. Is this something feasible or should I just follow the master slave pardigm and then have different blueprints for different environments ?

5 REPLIES 5

Re: documentation for blueprint processor

Rising Star

Hi @Anshuman Mehta,

A "host_group" defined in a Blueprint is mean to be mapped to a single host, or a set of hosts that are assumed to be identical (with respect to configuration and the components installed on each instance).

The "HOSTGROUP" token syntax mentioned in this question does not provide support for accessing a specific host within a set of hosts mapped to a host group. It's also important to note that the "HOSTGROUP" syntax is not a general purpose mechanism within Blueprints, and can only be used for configuration properties that are updated by the Blueprint configuration processor. With the exception of HA Blueprint deployments, the HOSTGROUP syntax can largely be avoided.

Hope this helps.

Thanks,

Bob

Re: documentation for blueprint processor

Expert Contributor

@rnettleton Thanks.

Is there a specific reason that in HA Blueprint deployments HOSTGROUP syntax may be important ?

The use case I was working towards was moving a blueprint across environments. For instance taking the simple case of HDFS HA with 3 nodes in my dev environment and say 20 odd nodes in production.

Then in order to specify the blueprint on my test environment for say

"dfs.namenode.shared.edits.dir" : "qjournal://%HOSTGROUP::master_1%:8485;%HOSTGROUP::master_2%:8485;%HOSTGROUP::master_3%:8485/HDFS"

my cluster template in the dev environment would consist of 3 masters which are also slaves. Clearly that is not the template for my production environment. I guess I can modify these files programmatically (ansible et all) and get what I need but I would need different cluster templates for each environment. Is this how blueprints are used by the community to progress from 1 environment to the next ?

Re: documentation for blueprint processor

Rising Star

Hi @Anshuman Mehta,

The support for Blueprint deployments of HA clusters was introduced back in the Ambari 2.0 timeframe, and the following link provides more information on this topic:

https://cwiki.apache.org/confluence/display/AMBARI/Blueprint+Support+for+HA+Clusters

The HOSTGROUP syntax for HA deployments is useful, since there is generally more configuration necessary for an HA cluster. For example: the configuration property names in the HDFS HA case will depend upon the name of the HA name service defined, and so the user needs to configure this manually. That being said, the Blueprint processor was updated to allow for usage of the HOSTGROUP syntax in these cases, in order to provide for making the Blueprint as portable as possible.

Generally, the Blueprint should be considered a logical definition of a cluster, with any environment-specific configuration residing in the cluster creation template. The property you mentioned could be configured differently in separate cluster creation templates, allowing the Blueprint itself to be more portable and re-usable in different environments.

Depending upon the cluster design, this property could be expanded for different cluster sizes. As you mentioned, the master-worker model is most commonly used in Ambari Cluster Deployments. If a given host group is matched to three HDFS datanode hosts in the development cluster, a production cluster could be represented in a separate cluster creation template that matches a greater number of hosts to this host group.

Configuration in a Blueprint can be overridden in the cluster creation template, and that is usually how environment-specific issues are handled in these deployments.

Hope this helps.

Thanks,

Bob

Re: documentation for blueprint processor

Expert Contributor

Thanks this indeed does help.

What this implies is that the scaling is really of the cluster design that is components per hostgroup. So if components are fixed per hostgroup then the blueprint design scales out.

I was looking for a model in which roles could be pushed to the nodes available in which case i really don't need a cluster design but based on my inventory of hosts, i could juggle or box them into roles and then push those roles onto them.(much akin to using the rest api to add the component of a service to the host)

For instance if in development i have 2 nodes where i pack in the namenode, datanode, zookeeper server components on one hostgroup and secondary_namenode, datanode, hbase_master, region_server on the other then this would be the cluster design for development. I would need a separate blueprint then for production which would for instance pack the namenode , hbase_master and secondary_namenode each on separate hostgroups . Is this correct ?

EDIT: I guess I could attempt to create multiple hostgroups with similar hosts for example a master1 group and slave1 group each with the same fqdn. Even if that works (topology validation i guess could fail) , an export of the blueprint would probably not honor the two hostgroups is my guess. Will try.

EDIT: Cannot have multiple nodes assigned to a hostgroup; so that will not work.

"Topology validation failed: org.apache.ambari.server.topology.InvalidTopologyException: The following hosts are mapped to multiple host groups:

Thanks !

Re: documentation for blueprint processor

@Anshuman Mehta: Did you achieve success in what you were trying? I have a similar use-case where I wan't multiple hosts/nodes in 'master' host-group and want to setup HA between components using that.