Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CloudBreak + HDP based on hardened images from azure marketplace

Solved Go to solution
Highlighted

CloudBreak + HDP based on hardened images from azure marketplace

New Contributor

I'm using CloidBreak for deploying hdp clusters. However the requirement is to do it based on hardened image that follows CIS guidelines, so I wanted to use their CentOS 7 image from Azure Marketplace. I did all prerequisites (packer built me an image, I've added new image catalog etc). However, as you can already suspect, the deployment failed because of that:

{
  "status": "Failed",
  "error": {
    "code": "ResourceDeploymentFailure",
    "message": "The resource operation completed with terminal provisioning state 'Failed'.",
    "details": [
      {
        "code": "VMMarketplaceInvalidInput",
        "message": "Creating a virtual machine from Marketplace image requires Plan information in the request. VM: '/subscriptions/xxxx/resourceGroups/xxxx/providers/Microsoft.Compute/virtualMachines/xxxxxxxm1'."
      }
    ]
  }

This is because ARM template does not have plan information, that is required when deploying VMs from marketplace. Unfortunately and super sadly I can't see a workaround for that unless I modify cloudbreak itself...

1 ACCEPTED SOLUTION

Accepted Solutions

Re: CloudBreak + HDP based on hardened images from azure marketplace

@Jakub Igla

Your use case is perfectly valid, but unfortunately Cloudbreak does not support it yet (it is in the roadmap though)

As a workaround, you can do the following:

  • Add this fragment to the Profile
export CB_JAVA_OPTS="-Dcb.arm.template.path=arm-v2.ftl"
  • Modify the file arm-v2.ftl relevant to the version of Cloudbreak you are using with the missing "Plan" parameters
  • Save the file on your Cloudbreak machine in the /var/lib/cloudbreak-deployment/etc directory.
  • Restart Cloudbreak

After restart has completed, all the subsequent cluster launches are to use your new, customized ARM template!

Hope this helps!

3 REPLIES 3

Re: CloudBreak + HDP based on hardened images from azure marketplace

@Jakub Igla

Your use case is perfectly valid, but unfortunately Cloudbreak does not support it yet (it is in the roadmap though)

As a workaround, you can do the following:

  • Add this fragment to the Profile
export CB_JAVA_OPTS="-Dcb.arm.template.path=arm-v2.ftl"
  • Modify the file arm-v2.ftl relevant to the version of Cloudbreak you are using with the missing "Plan" parameters
  • Save the file on your Cloudbreak machine in the /var/lib/cloudbreak-deployment/etc directory.
  • Restart Cloudbreak

After restart has completed, all the subsequent cluster launches are to use your new, customized ARM template!

Hope this helps!

Re: CloudBreak + HDP based on hardened images from azure marketplace

New Contributor

Hi @pdarvasi

I confirm, this workaround did the job for me. Thanks again.

Also, I saw on github, there is work being done on 2.8 and even 2.9.
Can I read somewhere about new features etc and maybe test a lil bit?

Re: CloudBreak + HDP based on hardened images from azure marketplace

New Contributor

Wow, this is exactly what I was trying to achieve. When I have a time I'll test this and let you know as for now I went different route.

Thanks!