CloudBreak + HDP based on hardened images from azure marketplace

[#1]

I'm using CloidBreak for deploying hdp clusters. However the requirement is to do it based on hardened image that follows CIS guidelines, so I wanted to use their CentOS 7 image from Azure Marketplace. I did all prerequisites (packer built me an image, I've added new image catalog etc). However, as you can already suspect, the deployment failed because of that:

{
  "status": "Failed",
  "error": {
    "code": "ResourceDeploymentFailure",
    "message": "The resource operation completed with terminal provisioning state 'Failed'.",
    "details": [
      {
        "code": "VMMarketplaceInvalidInput",
        "message": "Creating a virtual machine from Marketplace image requires Plan information in the request. VM: '/subscriptions/xxxx/resourceGroups/xxxx/providers/Microsoft.Compute/virtualMachines/xxxxxxxm1'."
      }
    ]
  }

This is because ARM template does not have plan information, that is required when deploying VMs from marketplace. Unfortunately and super sadly I can't see a workaround for that unless I modify cloudbreak itself...

[#2 · Accepted solution]

@Jakub Igla

Your use case is perfectly valid, but unfortunately Cloudbreak does not support it yet (it is in the roadmap though)

As a workaround, you can do the following:

• Add this fragment to the Profile

export CB_JAVA_OPTS="-Dcb.arm.template.path=arm-v2.ftl"

• Modify the file arm-v2.ftl relevant to the version of Cloudbreak you are using with the missing "Plan" parameters
• Save the file on your Cloudbreak machine in the /var/lib/cloudbreak-deployment/etc directory.
• Restart Cloudbreak

After restart has completed, all the subsequent cluster launches are to use your new, customized ARM template!

Hope this helps!

[#3]

Hi @pdarvasi

I confirm, this workaround did the job for me. Thanks again.

Also, I saw on github, there is work being done on 2.8 and even 2.9.
Can I read somewhere about new features etc and maybe test a lil bit?

[#4]

Wow, this is exactly what I was trying to achieve. When I have a time I'll test this and let you know as for now I went different route.

Thanks!