Setting IAM Role for AWS ec2 Instances using Cloudbreak 2.7

[#1]

Hi,

Requirement: Set IAM role for AWS ec-2 instances in the cluster.

Cloudbreak 2.4: Using cloudbreak 2.4 I used to create cluster with the following property in the template.

"parameters": {    

    "instanceProfile": "AWS_INSTANCE_PROFILE",    

    "instanceProfileStrategy": "USE_EXISTING"

}

This setting would create instances in the cluster with Role ARN of the Instance Profile as IAM Role.

Cloudbreak 2.7: In the process of recreating the clusters using cloudbreak 2.7, I am facing difficulties setting IAM Role for the instance.

I tried specifying instanceProfile like cloudbreak 2.4

"parameters": {    

    "instanceProfile": "AWS_INSTANCE_PROFILE",    

    "instanceProfileStrategy": "USE_EXISTING"

}

However the IAM Role was not set.

I created a HDP 3.0 cluster by selecting the CloudStorage option from UI and also saw that the IAM Role is not set.

Also I couldn't access S3 buckets that the role has access to.

I also tried creating a cluster using the following block inside "cluster" in the template.

"cloudStorage": {
  "s3": {
    "instanceProfile": "instance-profile-arn"
   }
}

However the IAM Role is not set for any instance in the cluster.

And I am not able to access S3 from any instance in the cluster.

Can you please let me know, how to set IAM Role for the instances in the cluster.

My requirement is to access S3 Locations from the cluster.

[#2]

cc: @pdarvasi@Dominika Bialek

[#3 · Accepted solution]

Hi @Rajkamal Mahamuni Natarajan,

Here is an example how the instance profile needs to look like:

"cloudStorage": {
      "locations": null,
      "s3": {
        "instanceProfile": "arn:aws:iam::1234567890:instance-profile/full-access"
      },

[#4]

Hi @rkovacs,

Thanks for the response. I used the same json block to set Instance Profile.

But the IAM Role for the ec2-instances in the cluster are not getting set.

I looked cloudbreak 2.7 release code, and the code to picks up instanceProfile.

I am still in the process of debugging cloudbreak 2.7 to see why IAM Role is not getting set.

[#5]

I am able to successfully set the IAM Role by creating a cluster from UI.

However, If i get the template for the cluster and use the template to create cluster using cb shell, the IAM Role is not getting set.

I will update the thread with progress.

[#6]

The IAM Role was not getting set as i was using cb shell 2.4 and not 2.7.

Once I downloaded the CLI 2.7, I was able to created a cluster with Iam Role set.

I used the Cloud Storage as mentioned by @rkovacs

Did a Blunder not upgrading CB shell !