Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Azure Cloudbreak - HOSTGROUP variables in Ambari Blueprint

Azure Cloudbreak - HOSTGROUP variables in Ambari Blueprint

New Contributor

Hi,

Cloudbreak 1.16.5
Azure
HDP 2.6.1 (to be upgraded)

We are trying to setup NameNode HA (High Availability) on Cloudbreak on top of Azure. For that we are using a few (standard) configurations during the blueprint like bellow:

"hdfs-site":

{ "properties" :

{ "....

"dfs.ha.namenodes.mycluster" : "nn1,nn2",

"dfs.namenode.http-address" : "%HOSTGROUP::master%:50070",
"dfs.namenode.http-address.mycluster.nn1" : "%HOSTGROUP::master%:50070",
"dfs.namenode.http-address.mycluster.nn2" : "%HOSTGROUP::compute%:50070",
"dfs.namenode.https-address" : "%HOSTGROUP::master_1%:50470",
"dfs.namenode.https-address.mycluster.nn1" : "%HOSTGROUP::master%:50470",
"dfs.namenode.https-address.mycluster.nn2" : "%HOSTGROUP::compute%:50470",
"dfs.namenode.rpc-address.mycluster.nn1" : "%HOSTGROUP::master%:8020",
"dfs.namenode.rpc-address.mycluster.nn2" : "%HOSTGROUP::compute%:8020",
"dfs.namenode.shared.edits.dir" : "qjournal://%HOSTGROUP::master_1%:8485;%HOSTGROUP::master_2%:8485;%HOSTGROUP::master_3%:8485/mycluster",
"dfs.nameservices" : "mycluster" }
}

Our blueprint is based on original HDP 2.6 Hive LLAP one.

The problem here is the %HOSTGROUP::HOST_GROUP_NAME% variable is not replaced during cluster creation with correct hostnames generated by Azure and the cluster fails to create in HA mode.
Also, Cloudbreak doesn't support Host Mapping configuration and we can't assign a specific hostname or IP to a hostgroup name. Further mode, we can't specify fixed IP address for the blueprint NameNode HA configuration because Azure will random assign different IPs to the nodes.
Can somebody confirm that %HOSTGROUP variables are NOT supported on Cloudbreak 1.16.5? Or do we have a flaw in our configuration that is preventing the processor to replace the variables.
Any suggestion? We are mostly stuck with this.
3 REPLIES 3
Highlighted

Re: Azure Cloudbreak - HOSTGROUP variables in Ambari Blueprint

Expert Contributor

Hi,

I don't see your blueprint so I cannot check whether the hostnames match or not, I used this NN HA blueprint, it might help to figure out what is missing. Namenode HA BP

Re: Azure Cloudbreak - HOSTGROUP variables in Ambari Blueprint

New Contributor

Hi,

We solved the issue. We had an error in the blueprint (two RESOURCEMANAGER, instead of one).

Conclusion is: If you have an error in your blueprint, you may not get the values of %HOSTGROUP..% replaced. Instead of relying on cloudbreak UI, you should check ambari server logs when blueprint deployment starts.

Thank you.

Re: Azure Cloudbreak - HOSTGROUP variables in Ambari Blueprint

New Contributor

@Luis Marques What did you put in place of "mycluster" in your blueprint snippet above?

Don't have an account?
Coming from Hortonworks? Activate your account here