question Re: CloudBreak + HDP based on hardened images from azure marketplace in Archives of Support Questions (Read Only)

CloudBreak + HDP based on hardened images from azure marketplace

jakub_igla1 — Sat, 18 Aug 2018 15:33:21 GMT

I'm using CloidBreak for deploying hdp clusters. However the requirement is to do it based on hardened image that follows CIS guidelines, so I wanted to use their CentOS 7 image from Azure Marketplace. I did all prerequisites (packer built me an image, I've added new image catalog etc). However, as you can already suspect, the deployment failed because of that:

{
  "status": "Failed",
  "error": {
    "code": "ResourceDeploymentFailure",
    "message": "The resource operation completed with terminal provisioning state 'Failed'.",
    "details": [
      {
        "code": "VMMarketplaceInvalidInput",
        "message": "Creating a virtual machine from Marketplace image requires Plan information in the request. VM: '/subscriptions/xxxx/resourceGroups/xxxx/providers/Microsoft.Compute/virtualMachines/xxxxxxxm1'."
      }
    ]
  }

This is because ARM template does not have plan information, that is required when deploying VMs from marketplace. Unfortunately and super sadly I can't see a workaround for that unless I modify cloudbreak itself...

Re: CloudBreak + HDP based on hardened images from azure marketplace

darvasip — Wed, 22 Aug 2018 18:26:06 GMT

@Jakub Igla

Your use case is perfectly valid, but unfortunately Cloudbreak does not support it yet (it is in the roadmap though)

As a workaround, you can do the following:

Add this fragment to the Profile

export CB_JAVA_OPTS="-Dcb.arm.template.path=arm-v2.ftl"

Modify the file arm-v2.ftl relevant to the version of Cloudbreak you are using with the missing "Plan" parameters
Save the file on your Cloudbreak machine in the /var/lib/cloudbreak-deployment/etc directory.
Restart Cloudbreak

After restart has completed, all the subsequent cluster launches are to use your new, customized ARM template!

Hope this helps!

Re: CloudBreak + HDP based on hardened images from azure marketplace

jakub_igla1 — Wed, 22 Aug 2018 21:26:09 GMT

Wow, this is exactly what I was trying to achieve. When I have a time I'll test this and let you know as for now I went different route.

Thanks!

Re: CloudBreak + HDP based on hardened images from azure marketplace

jakub_igla1 — Sun, 09 Sep 2018 03:19:37 GMT

Hi @pdarvasi

I confirm, this workaround did the job for me. Thanks again.

Also, I saw on github, there is work being done on 2.8 and even 2.9.
Can I read somewhere about new features etc and maybe test a lil bit?