Cloudbreak is executing Post type recipe before all cluster services have been installed

[#1]

Cloudbreak 1.14.0

OpenStack

I created a new Post type recipe. I have done this many times before and up until today, Post recipes would start execution after the Ambari Blueprint is completely finished installing. That is, all services in Amabri are fully installed and started. For some reason, this recipe begins execution as soon as Ambari starts installing the Blueprint. I am aware that Cloudbreak 1.14.0 is TP but I have been using this version for the last month. Is there something special that is needs to happen to ensure that a Post type recipe begins execution after all Ambari services have completed execution?

[#2 · Accepted solution]

Hi,

Would you mind sharing your logs with us? Post recipes should run after the install. If you look at the logs there should be a message that says: "Submitted cluster creation template" then it polls the installation progress then executes the post recipes. This part is the relevant to see what happened. Maybe we got a 100 percent message while the installation is still in progress then the flow moved on, logs can help identify this.

[#3]

cbd-log.txt

@khorvath

That's exactly what happened. Any idea why?

[#4]

Maybe there are some exceptions or error messages in the ambari-server.log. I think Ambari reports 100% when something went wrong and skipped the installation or some steps. Can you check please?

[#5]

@khorvath

I dug into the Ambari logs as you suggested. There is nothing obvious but Ambari is definitely returning success/complete as it begins to install services. I probably should have led with this, but, I am using early bits from Ambari 2.5.1 (the version where HDP and HDF can be installed in the same cluster). There is probably some sort of disconnect between what Salt is looking far and what Ambari is actually returning. Perhaps this is already being addressed in CB 1.15.0. Thanks your your help.