Member since
10-02-2015
50
Posts
12
Kudos Received
11
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
665 | 01-03-2018 04:12 PM | |
548 | 01-03-2018 04:07 PM | |
741 | 07-20-2017 02:18 PM | |
986 | 06-22-2017 09:33 PM | |
413 | 03-20-2017 02:57 PM |
01-03-2018
04:12 PM
Hi @Sampath Kumar, Some changes in Ambari 2.6.0 have been introduced that modify the way that install repository overrides are setup. Please see this link that describes the new process for setting up local repos: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.0.0/bk_ambari-release-notes/content/ambari_relnotes-2.6.0.0-behavioral-changes.html Hope this helps. Thanks, Bob
... View more
01-03-2018
04:07 PM
Hi @Anshuman Mehta, 1. That's correct, Blueprints are only for the initial install of a cluster. 2. Incremental changes cannot be applied to a Blueprint after a cluster has been created. A Blueprint, once posted to the REST APIs, is considered a static document. Updating a Blueprint after a cluster has already been created with that Blueprint will have no affect on the existing cluster. 3. Yes, REST calls would need to be made in order to update the configuration for the existing cluster. 4. Generally, the Blueprint export feature can be used to take a snapshot of the existing cluster ( components and configuration), and then re-use that Blueprint to create a new cluster, perhaps making some modifications prior to deploying the new cluster. It is also worth noting that Blueprint exports can be taken on clusters that were created with the Ambari UI, and this can provide a way to reproduce an existing cluster, even if it was not originally created with Blueprints. While not directly related to your question, I thought it would be useful to mention that Blueprints integration with the Ambari StackAdvisor was introduced in Ambari 2.2.0, and can be used to apply Ambari's configuration recommendations to the Blueprint specified. Here's the link that provides more information on that topic: https://cwiki.apache.org/confluence/display/AMBARI/Blueprints#Blueprints-ClusterCreationTemplateStructure Thanks, Bob
... View more
01-03-2018
03:58 PM
Hi @Dima Kovalyov, The "HOSTGROUP" token syntax is not a general-purpose mechanism for resolving hostnames in the configuration for a Blueprint. This syntax is generally used in the following scenarios: 1. Exported Blueprints 2. HA-related Blueprints Outside of these scenarios, the "HOSTGROUP" syntax is not recommended. The Ambari Blueprints configuration processor will attempt to match HOSTGROUP tokens with hostnames for a specific subset of configuration properties, but this subset of properties is a static list, and is not configurable. As you mentioned, it is also true that out-of-stack services are not supported by this syntax, for the reason I've listed above. The "{{ var }}" syntax is not directly supported by the Blueprints processor. There might be cases where a configuration property is pushed down to an ambari-agent that interprets this python-like variable, but this behavior is not guaranteed to work, and should be avoided in Blueprints. Hope this helps. Thanks, Bob
... View more
01-03-2018
03:41 PM
Hi @Anshuman Mehta, The support for Blueprint deployments of HA clusters was introduced back in the Ambari 2.0 timeframe, and the following link provides more information on this topic: https://cwiki.apache.org/confluence/display/AMBARI/Blueprint+Support+for+HA+Clusters The HOSTGROUP syntax for HA deployments is useful, since there is generally more configuration necessary for an HA cluster. For example: the configuration property names in the HDFS HA case will depend upon the name of the HA name service defined, and so the user needs to configure this manually. That being said, the Blueprint processor was updated to allow for usage of the HOSTGROUP syntax in these cases, in order to provide for making the Blueprint as portable as possible. Generally, the Blueprint should be considered a logical definition of a cluster, with any environment-specific configuration residing in the cluster creation template. The property you mentioned could be configured differently in separate cluster creation templates, allowing the Blueprint itself to be more portable and re-usable in different environments. Depending upon the cluster design, this property could be expanded for different cluster sizes. As you mentioned, the master-worker model is most commonly used in Ambari Cluster Deployments. If a given host group is matched to three HDFS datanode hosts in the development cluster, a production cluster could be represented in a separate cluster creation template that matches a greater number of hosts to this host group. Configuration in a Blueprint can be overridden in the cluster creation template, and that is usually how environment-specific issues are handled in these deployments. Hope this helps. Thanks, Bob
... View more
01-02-2018
10:24 PM
Hi @Anshuman Mehta, A "host_group" defined in a Blueprint is mean to be mapped to a single host, or a set of hosts that are assumed to be identical (with respect to configuration and the components installed on each instance). The "HOSTGROUP" token syntax mentioned in this question does not provide support for accessing a specific host within a set of hosts mapped to a host group. It's also important to note that the "HOSTGROUP" syntax is not a general purpose mechanism within Blueprints, and can only be used for configuration properties that are updated by the Blueprint configuration processor. With the exception of HA Blueprint deployments, the HOSTGROUP syntax can largely be avoided. Hope this helps. Thanks, Bob
... View more
07-20-2017
02:18 PM
1 Kudo
Hi @Ali, There is a way to have the Blueprints processor apply the Ambari Server recommendations for configuration for various services. In Ambari 2.2.0, Blueprints was updated to provide integration with the StackAdvisor/Recommendations engine. Basically, adding some configuration in the cluster creation template will instruct the Blueprints processor to apply the output of the recommendations engine. There are various strategies that can be applied, based on the desired outcome. By default, the strategy is "NEVER_APPLY", which means that recommendations are not applied by default. This was done to ensure backwards compatibility, as the Blueprints processor preceded the StackAdvisor implementation in Ambari. Here's a link to the configuration you'll need to add to your cluster creation template if you'd like to have Blueprints apply the recommendations automatically: https://cwiki.apache.org/confluence/display/AMBARI/Blueprints#Blueprints-ClusterCreationTemplateStructure The "config_recommendation_strategy" property needs to be added to the cluster creation template, and set to one of the supported strategies. This should allow you to make your Blueprint smaller and more portable, since the recommended values do not need to be set, unless you prefer to override the values set by the recommendations engine. Hope this helps. Thanks, Bob
... View more
06-26-2017
03:26 PM
Hi @Kirk DeMumbrane, Glad this information was useful. If this resolves your problem, can you please accept this answer? Thanks, Bob
... View more
06-22-2017
09:33 PM
1 Kudo
Hi @Kirk DeMumbrane, Can you post what version of Ambari you're using? In the LogSearch Technical Preview, which was introduced in Ambari 2.4, there was a bug that would keep the Ambari integration layer (which displays the tail log of components in Ambari itself) from working properly when SSL was enabled. This issue was resolved in Ambari 2.5: https://issues.apache.org/jira/browse/AMBARI-19104 If you are working with Ambari 2.5, then you can probably just update the "logsearch_ui_protocol" in the "logsearch-env" configuration type to be "https", restart ambari-server, and the connection between ambari-server and the LogSearch Portal should work properly at that point, and I would expect that the error logs in ambari-server.log would stop occurring. Regarding the "No Data Available" message, it is possible that the existing logging data in your cluster has expired, and been removed from the Solr data store. Typically, Solr expires logging data from LogSearch after 7 days, unless this has been explicitly configured. Hope this helps! Bob
... View more
06-13-2017
08:41 PM
Hi @sarath chandra, Based on the exception you provided, it looks like ambari-server cannot connect to the Postgres database that Ambari relies on. Can you check your postgresql database, to make sure it is up and running? Hope this helps, Bob
... View more
04-13-2017
06:35 PM
Hi @Theyaa Matti, Depending upon the services you have deployed in your cluster, the audit logs will generally be written to for service-specific actions that occur (HDFS write, HDFS read, Ambari REST calls, etc). Are you looking for a specific service's audit logs? Please note that not all services write audit logs. What version of Ambari are you using? Hope this helps, Bob
... View more
03-20-2017
02:57 PM
Hi @Vladislav Falfushinsky, It is expected behavior that you would receive a Blueprint with some differences from the original Blueprint that you created the cluster with. It's important to note that the Blueprint "export" feature works for any cluster deployed by Ambari, including those created via the Ambari Web UI Wizard. This means that the exported Blueprint will give you the current state of the running cluster, whether or not this cluster was created with a Blueprint originally. This is why the host group names are different, since the Blueprint exporter needs to generate names automatically. The configuration may be different as well. There are a few reasons for this: 1. Configuration may have changed between the time the Blueprint cluster was deployed and before the export was taken, 2. The Blueprint export will include all configuration in the current cluster in the Blueprint. This was implemented in this way in order to give a complete snapshot of the cluster at the time of export. This of course will differ from the original Blueprint (if one was used), since likely only configuration overrides are included in the Blueprint. Another possible area of difference is that Blueprints will sometimes add dependency components (typically client components) to a Blueprint deployment at Cluster creation time. If validate_topology is set to "false" then these additions won't occur, but they are turned on by default. If you need to see the Blueprint you used to create a cluster, you can always access it via the REST "blueprints/$BLUEPRINT_NAME" resource, where "$BLUEPRINT_NAME" is the name of your Blueprint. Comparing the Blueprint you originally used with with an exported one from the same cluster at a later point in time may not be feasible. Hope this helps. Bob
... View more
03-03-2017
04:38 PM
Hi @christophe menichetti, As @Predrag Monodic mentioned, you can use Blueprints for non-UI based installs. Unfortunately, the UI Wizard will not allow you to generate a Blueprint and Cluster Creation template after you gone through all the screens. The simplest way to generate a Blueprint to start with is to try the following: 1. On a local VM cluster for testing (vagrant, docker, etc), create a cluster that has the services, components, and configuration that you are interested in deploying in your production cluster. 2. Use the UI to deploy this local cluster, going through all the normal screens in the wizard. 3. You can then export the Blueprint from this running cluster. This REST call will generate a Blueprint based on the currently-running cluster you setup in Step #2. 4. Save this Blueprint, and customize it as necessary. 5. Create a Cluster Creation Template that matches hostnames to the host groups from the exported Blueprint. Please note that you may want to manually rename the host groups in the exported Blueprint, as they are generated using a "host_group_n" convention, which may not be useful for documenting your particular cluster. You can check out the following link on the Blueprints wiki to see how to make the REST call to export the Blueprint from a running cluster: https://cwiki.apache.org/confluence/display/AMBARI/Blueprints#Blueprints-APIResourcesandSyntax Hope this helps!
... View more
02-15-2017
04:30 PM
HI @Vladislav Falfushinsky, The Blueprints processor does validate dependencies in a Blueprint, but at a different level than the UI. The Ambari Web UI will validate at the service level (HDFS, Yarn, etc), while the Blueprint validates the dependencies between components ("NAMENODE", "YARN_RESOURCEMANAGER", etc). This is due to the fact that Blueprints operate only on components, and not at the service level. This is why you see slightly different validation behavior in either deployment mode. The Blueprint processor validates a Blueprint's components for dependencies using the dependencies defined in the stack definition for that service or set of components. There are two possible reasons why you might not see the dependencies being verified: 1. If you've turned off validation while POST-ing the Blueprint, then no component-level validation will occur. This validation is on by default, and is controlled by a query parameter that can be set when POST-ing the Blueprint: https://cwiki.apache.org/confluence/display/AMBARI/Blueprints#Blueprints-BlueprintUsageOverview 2. If the stack definitions for the services in your Blueprint do not correctly specify dependencies between components, then validation may not occur, as the Blueprint validation process is driven by the stack data. Hope this helps.
... View more
02-09-2017
06:34 PM
Hi @Vladislav Falfushinsky, Apart from the validation routines that you mentioned at the Blueprint and Cluster Creation Template levels, there isn't really much support for a "dry run" deployment, although that would be a good idea for a future release, since it would assist in some types of testing while developing a new Blueprint. I'm not sure what version of Ambari is being used in your case, but I'll assume Ambari 2.4 for now. In Ambari 2.1.0, the Blueprints deployment mechanism was refactored to allow for a more dynamic deployment model, such that hosts could join the cluster after the Blueprint deployment started, and the Blueprints processor will react to this and update the configuration accordingly. I'm not sure how you created the Blueprint, but if you exported a Blueprint from a particular cluster, and are trying to recreate the cluster on a slightly different topology, or using a different set of services, that may be a part of the problem. Can you post the portion of the ambari-server.log that shows the error you mentioned? The host group involved in the failure should be mentioned in the log message. Can you also post the Blueprint you're trying to deploy? It may be that your Blueprint references a host group that is not defined in the Blueprint, or perhaps a "%HOSTGROUP%" token is being used for a component that is not included in any host groups in your Blueprint. Hope this helps.
... View more
02-08-2017
04:02 PM
Hi @Navdeep Singh, Did this resolve your Blueprint deployment issue? If so, can you please accept this answer, in case it might be useful for others as well? Thanks
... View more
02-06-2017
04:57 PM
Hi @Navdeep Singh, The problem you're experiencing is an issue with configuration. I tried out the Blueprint you've posted here with a local vagrant cluster, and there were no exceptions in the ambari-server log, which seems to indicate that the Blueprint deployment completed without any unexpected errors. I looked through the HBase logs, and noticed some exceptions when HBase attempts to connect to HDFS. Generally, HBase must be configured to point towards the HDFS Namenode being used. In your particular case, the configuration was pointing to a "localhost" address, which is incorrect, since the NameNodes in your cluster are not deployed on the same instance as either of the HBase Master components. If you add the following configuration block to your Blueprint, the deployment will work fine: {
"hbase-site" : {
"properties" : {
"hbase.rootdir" : "hdfs://hdpcluster/apps/hbase/data"
}
}
} Since your HDFS configuration defines a name service at "hdpcluster", this must be used in the HBase configuration that points to the root directory in HDFS used by HBase. I modified a local copy of your Blueprint with this change added, and the cluster deployed properly, and HBase started up as expected (Active and Standby nodes started, regsionservers started). For Blueprint deployments, the HDFS HA settings are not automatically set by the Blueprints processor for services/components that depend upon HDFS. This should probably be updated in a future version of Ambari. I've included a modified copy of your Blueprint below that seems to work fine now with the new configuration. Hope this helps. {
"configurations" : [
{
"hdfs-site" : {
"properties" : {
"dfs.namenode.https-address" : "%HOSTGROUP::master_1%:50470",
"dfs.client.failover.proxy.provider.hdpcluster" : "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider",
"dfs.namenode.rpc-address.hdpcluster.nn2" : "%HOSTGROUP::master_2%:8020",
"dfs.namenode.shared.edits.dir" : "qjournal://%HOSTGROUP::master_1%:8485;%HOSTGROUP::slave_1%:8485;%HOSTGROUP::master_2%:8485/hdpcluster",
"dfs.namenode.http-address.hdpcluster.nn2" : "%HOSTGROUP::master_2%:50070",
"dfs.namenode.http-address.hdpcluster.nn1" : "%HOSTGROUP::master_1%:50070",
"dfs.ha.fencing.methods" : "shell(/bin/true)",
"dfs.nameservices" : "hdpcluster",
"dfs.namenode.http-address" : "%HOSTGROUP::master_1%:50070",
"dfs.ha.namenodes.hdpcluster" : "nn1,nn2",
"dfs.namenode.https-address.hdpcluster.nn2" : "%HOSTGROUP::master_2%:50470",
"dfs.namenode.rpc-address.hdpcluster.nn1" : "%HOSTGROUP::master_1%:8020",
"dfs.namenode.https-address.hdpcluster.nn1" : "%HOSTGROUP::master_1%:50470",
"dfs.ha.automatic-failover.enabled" : "true"
}
}
},
{
"core-site" : {
"properties" : {
"fs.defaultFS" : "hdfs://hdpcluster",
"ha.zookeeper.quorum" : "%HOSTGROUP::master_1%:2181,%HOSTGROUP::master_2%:2181,%HOSTGROUP::slave_1%:2181"
}
}
},
{
"hbase-site" : {
"properties" : {
"hbase.rootdir" : "hdfs://hdpcluster/apps/hbase/data"
}
}
}
],
"host_groups" : [
{
"components" : [
{
"name" : "HDFS_CLIENT"
},
{
"name" : "MAPREDUCE2_CLIENT"
},
{
"name" : "YARN_CLIENT"
},
{
"name" : "ZOOKEEPER_SERVER"
},
{
"name" : "HISTORYSERVER"
},
{
"name" : "TEZ_CLIENT"
},
{
"name" : "ZKFC"
},
{
"name" : "JOURNALNODE"
},
{
"name" : "NAMENODE"
},
{
"name" : "ZOOKEEPER_CLIENT"
},
{
"name" : "METRICS_MONITOR"
}
],
"configurations" : [ ],
"name" : "master_1",
"cardinality" : "1"
},
{
"components" : [
{
"name" : "NAMENODE"
},
{
"name" : "TEZ_CLIENT"
},
{
"name" : "RESOURCEMANAGER"
},
{
"name" : "MAPREDUCE2_CLIENT"
},
{
"name" : "ZKFC"
},
{
"name" : "YARN_CLIENT"
},
{
"name" : "METRICS_GRAFANA"
},
{
"name" : "ZOOKEEPER_CLIENT"
},
{
"name" : "ZOOKEEPER_SERVER"
},
{
"name" : "METRICS_MONITOR"
},
{
"name" : "APP_TIMELINE_SERVER"
},
{
"name" : "HDFS_CLIENT"
},
{
"name" : "JOURNALNODE"
},
{
"name" : "METRICS_COLLECTOR"
}
],
"configurations" : [ ],
"name" : "master_2",
"cardinality" : "1"
},
{
"components" : [
{
"name" : "ZOOKEEPER_CLIENT"
},
{
"name" : "YARN_CLIENT"
},
{
"name" : "DATANODE"
},
{
"name" : "HBASE_MASTER"
},
{
"name" : "MAPREDUCE2_CLIENT"
},
{
"name" : "HBASE_REGIONSERVER"
},
{
"name" : "METRICS_MONITOR"
},
{
"name" : "HBASE_CLIENT"
},
{
"name" : "HDFS_CLIENT"
},
{
"name" : "TEZ_CLIENT"
},
{
"name" : "NODEMANAGER"
}
],
"configurations" : [ ],
"name" : "slave_2",
"cardinality" : "1"
},
{
"components" : [
{
"name" : "HDFS_CLIENT"
},
{
"name" : "HBASE_CLIENT"
},
{
"name" : "YARN_CLIENT"
},
{
"name" : "HBASE_MASTER"
},
{
"name" : "TEZ_CLIENT"
},
{
"name" : "NODEMANAGER"
},
{
"name" : "ZOOKEEPER_CLIENT"
},
{
"name" : "DATANODE"
},
{
"name" : "ZOOKEEPER_SERVER"
},
{
"name" : "METRICS_MONITOR"
},
{
"name" : "JOURNALNODE"
},
{
"name" : "HBASE_REGIONSERVER"
},
{
"name" : "MAPREDUCE2_CLIENT"
}
],
"configurations" : [ ],
"name" : "slave_1",
"cardinality" : "1"
}
],
"settings" : [ ],
"Blueprints" : {
"blueprint_name" : "blueprint992c1c9a-b38d-4a8f-bb6f-b4e45f7f447d",
"stack_name" : "HDP",
"stack_version" : "2.5",
"security" : {
"type" : "NONE"
}
}
}
... View more
02-06-2017
04:21 PM
Hi @Yurii Tymchii, Are you trying to create a new cluster with specific recovery/restart settings? If so, the following wiki doc link might be helpful: https://cwiki.apache.org/confluence/display/AMBARI/Recovery%3A+auto+start+components#Recovery:autostartcomponents-240Onwards It is possible to configure the "recovery_enabled" settings via a Blueprint deployment. I believe that this will work on a per-service or per-component basis, depending upon the needs of your cluster deployment. The "REST API" section of the document mentioned above might also be useful, if you are trying to update the recovery settings on an existing cluster using the Ambari REST API.
... View more
01-17-2017
05:43 PM
1 Kudo
Hi @Kuldeep Kulkarni, Generally, it is good practice to avoid setting passwords in the Blueprint document. We generally recommend that any passwords be configured in the Cluster Creation Template document, which is POST-ed to actually create the cluster, based on a given Blueprint. Since the Cluster Creation Template is not persisted by Ambari, it is usually the best place to configure passwords. While not a perfect solution, since the document still includes the passwords in clear text, it does have the advantage of keeping the passwords out of the Blueprint, which is persisted by Ambari, and is available via the REST API (although the "secret reference" feature usually guarantees that the passwords are not available to a REST client). Hope this helps.
... View more
12-20-2016
10:15 PM
Hi @wbu, @Alejandro Fernandez's and @smagyari's are both correct. The main problem is that SmartSense's Ambari Stack definitions are not included in the default stack definitions. Generally, the configuration properties that are passwords are marked with metadata to indicate that a given property is of type "PASSWORD". This metadata is used by the Blueprints processor in order to determine which properties can be set with the value of "default_password", which is set in the Cluster Creation Template. In the current release (Ambari 2.4), the only way to resolve this would be to use @smagyari's recommendation, and set the password directly. Generally, we recommend that passwords only be included in the Cluster Creation Template, since Ambari does not persist that document. Hope this helps!
... View more
08-26-2016
07:34 PM
Hi @Raghu Udiyar, While the Blueprint POST-ed to Ambari will not change as the state of a cluster changes, you can consider "exporting" a Blueprint from a live cluster. After a set of changes to the cluster (Config changes, services added/deleted, etc), exporting the Blueprint will provide a Blueprint that describes the current state of the cluster, which would be different than the original Blueprint used to create the cluster. Based on what I've seen in this issue, it looks like you could use the Blueprint export feature to maintain a Blueprint of the current changes, so that you could always recreate this cluster layout on different hardware if necessary. Here's a link to the Blueprints documentation on Blueprint exports: https://cwiki.apache.org/confluence/display/AMBARI/Blueprints#Blueprints-APIResourcesandSyntax Blueprints can also be used to create HA-based clusters from scratch. That feature has been present since Ambari 2.0, and a link to more documentation on this can be found at: https://cwiki.apache.org/confluence/display/AMBARI/Blueprint+Support+for+HA+Clusters
... View more
04-08-2016
01:13 PM
2 Kudos
Hi @Jan Andreev, I'm not exactly sure why you're seeing the issue with automatic registration, but if you're just trying to get things up and running manually, you can probably just modify the ambari-agent config files. You manual setup step may have failed if the agents are not configured to point to the ambari-server instance in your cluster. You mentioned that the registration doesn't fail on the ambari-server node, and that makes sense, since the default hostname pointer is "localhost". For manual configuration, you can just set this up in the ambari-agent config file, on Centos 6 it will be in: /etc/ambari-agent/conf/ambari-agent.ini Just set the "hostname" property to be the DNS name for the node that has ambari-server running, and restart your ambari-agent instances. Hope this helps! Bob
... View more
04-06-2016
02:46 PM
1 Kudo
Hi @Anna Shaverdian, In addition to what has already been posted, I wanted to mention something about Blueprint exports. In general, you should be able to re-use an exported Blueprint from a running cluster without any changes. There are a few exceptions,however: 1. Passwords: Any password properties are filtered out of the exported Blueprint, and must be added in explicitly during a redeployment attempt. 2. External Database Connections: I'm defining an "External" DB connection as any DB that is used by the cluster, but is not managed by Ambari. The DB instance used by Ranger is one example, but Oozie and Hive can also use separate instances that are not managed by Ambari. In these cases, the Blueprint export process filters out these properties, since they are not necessarily portable to the new environment. From what I've seen in this posting, these are the kinds of properties that you'll need to add back into your Blueprint or Cluster Creation template, as the other posts have indicated. Hope this helps, Thanks, Bob
... View more
03-31-2016
01:20 PM
1 Kudo
This approach should work fine, but I would suggest a refinement that may improve performance somewhat, and will also make the returned status play load quite a bit smaller in size: If you use the partial response syntax provided in the Ambari REST API, you can filter out much of the data returned in the request resource returned by the call listed above. An example of using the partial response syntax is below: http://ambari-hostname:ambari-port-number/api/v1/clusters/clusterone/requests/1?fields=Requests/completed_task_count,Requests/request_status,Requests/task_count,Requests/failed_task_count The "fields" query parameter is used to limit the fields returned from the request resource. The fields I've mentioned here are the set I use, but you can also check the other properties returned by the resource, if a particular property is more straightforward to use for this type of monitoring. I use this syntax quite a bit when I want to monitor the status of a Blueprint deployment.
... View more
01-05-2016
08:36 PM
The comment from jspeidel is correct. If Oozie HA is being used in this Blueprint, then "oozie.base.url" must be set explicitly to the address of the loadbalancer being used, since Oozie HA requires a separate loadbalancer that is external to each Oozie instance. If you are just testing out your Blueprint, then you can just set this property to be the address of one or the other Oozie instances in your cluster. Here's a great reference on Oozie HA, that will be helpful in setting up an Oozie HA Blueprint: https://oozie.apache.org/docs/4.1.0/AG_Install.html#High_Availability_HA
... View more
12-17-2015
04:43 PM
1 Kudo
@hkropp, @Ali Bajwa, It is possible to define configuration groups when using a Blueprints install. Configuration in Blueprints can be specified at almost any level (cluster, blueprint, host group). A "host group" in Blueprints defines a set of components and configuration that can be applied to a host or group of hosts. These actual hosts are mapped in the Cluster Creation Template. You can specify configuration at the level of "host groups" in the Blueprint or Cluster Creation Template, and this has the affect of only applying these configuration overrides to the machine or group of machines that are mapped to this host group at deployment time. Specifying configuration at the "host group" level will cause the Ambari Blueprints processor to create Ambari configuration groups to manage that host-specific configuration. More information on Blueprint syntax and how configurations are applied may be found at: https://cwiki.apache.org/confluence/display/AMBARI/Blueprints#Blueprints-BlueprintStructure
... View more
12-17-2015
04:30 PM
@Vijay Srinivasaraghavan, Once a Blueprint is posted, the document is immutable, so there is no way currently to add a host group to a Blueprint. That being said, I believe you can obtain the same effect by trying the following:
POST a Blueprint that contains: the host groups used in your cluster, and an extra host group that only contains the "DATANODE" component (and perhaps any required dependencies). POST the Cluster Creation template as normal, but do not map any hosts to the host group that only contains "DATANODE" components. When necessary, use the "Add Host" API to add new hosts based on the "extra" host group. As long as there are no configuration dependencies specified that point to the host group that is not initially used, this should work fine. Usually, configuration dependencies would mean things such as %HOSTGROUP% references that refer to the "extra" host group, which shouldn't be necessary anyway. While this approach doesn't allow you to add host groups dynamically, it should allow you to have more fine-grained control over the types of components that are deployed during a scale-out operation.
... View more
10-27-2015
03:21 PM
1 Kudo
As Sumit mentioned, we currently do not use the StackAdvisor output in Blueprints, but will support this in a future release. If you have the hardware to experiment with, you can try deploying a cluster with the UI (which will cause the recommendations to be applied), and then export the Blueprint from the running cluster. You can then use the Kafka config in the exported Blueprint as a starting point. The Blueprints wiki contains the REST call used for exporting a Blueprint: https://cwiki.apache.org/confluence/display/AMBARI/Blueprints#Blueprints-APIResourcesandSyntax
... View more