Using blueprints for cluster WITHOUT a namenode

[#1]

I am trying to automate a cluster deployment for Kafka. I am able do this using the Ambari UI however I am having issues with blueprints.

Here is how I deployed using the Web UI

When deploying via the blueprint API I am getting an error about not having a NAMENODE. ( see full error at bottom of post) I first tried this with a handcrafted blueprint file. I went back and configured using the Ambari webUI and exported the blue print and got the same error.

Can anyone suggest a way of working around this issue?

Here is the blueprint I am using

{

"configurations" : [ ],

"host_groups" : [

{ "name" : "host_group_1",

"components" : [

{ "name" : "ZOOKEEPER_SERVER" },

{ "name" : "ZOOKEEPER_CLIENT" },

{ "name" : "KAFKA_BROKER" } ],

"cardinality" : "3" },

{ "name" : "host_group_2",

"components" : [

{ "name" : "METRICS_COLLECTOR" },

{ "name" : "METRICS_GRAFANA" },

{ "name" : "METRICS_MONITOR" },

{ "name" : "ZOOKEEPER_CLIENT" }],

"cardinality" : "1" }

],

"Blueprints" : {

"blueprint_name" : "blueprint-kafka01",

"stack_name" : "HDP",

"stack_version" : "2.4" } }

Full error:

02 Jun 2016 15:10:43,768 ERROR [pool-9-thread-1] TopologyManager:775 - TopologyManager.ConfigureClusterTask: An exception occurred while attempting to process cluster configs and set on cluster:

java.lang.IllegalArgumentException: Unable to update configuration property 'xasecure.audit.destination.hdfs.dir' with topology information. Component 'NAMENODE' is mapped to an invalid number of hosts '0'.

at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor$SingleHostTopologyUpdater.updateForClusterCreate(BlueprintConfigurationProcessor.java:1292)

at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doUpdateForClusterCreate(BlueprintConfigurationProcessor.java:257)

at org.apache.ambari.server.topology.ClusterConfigurationRequest.process(ClusterConfigurationRequest.java:139)

at org.apache.ambari.server.topology.TopologyManager$ConfigureClusterTask.call(TopologyManager.java:773)

at org.apache.ambari.server.topology.TopologyManager$ConfigureClusterTask.call(TopologyManager.java:747)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

[#2]

Filed a bug to track this issue. Will investigate and update this thread.

[#3]

It may be related to this dependency,

https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.3/role_com...

/var/lib/ambari-server/resources/stacks/HDP/2.3/role_command_order.json

"KAFKA_BROKER-START" : ["ZOOKEEPER_SERVER-START", "RANGER_USERSYNC-START", "NAMENODE-START"]

[#4]

Whats off is that using the web app doesn't have the issue. I was always under the impression was that blueprints was just automating the same rest calls the webui was calling.

It seems in this case blueprints is doing some additional validation the webui is not doing .

how is the role_command_order.json file being used?

Bert