Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cluster creation failed on Cloudbreak hosted on AWS

Cluster creation failed on Cloudbreak hosted on AWS

New Contributor

Hi,

I followed the exact steps to launch Cloudbreak (v2.7.0) on AWS :

https://hortonworks.github.io/cloudbreak-documentation/latest/aws-launch/index.html

As a pre-requisite, below two roles were created (based on AssumeRole and cb-policy json files as mentioned):

CloudbreakRole: Allows Cloudbreak to assume other IAM roles - specifically the CredentialRole.
CredentialRole: Allows Cloudbreak to create AWS resources required for clusters.

(Referring to https://hortonworks.github.io/cloudbreak-documentation/latest/aws-pre/index.html)

I could successfully launch Cloudbreak and create a Cloudbreak credential.

I used "Role based" authentication to create cluster. Used platform as HDP 2.6 and blueprint: "EDW-ETL: Apache Hive, Apache Spark 2" with 2 nodes.
However, cluster creation is failing with the below errors:


java.util.concurrent.ExecutionException: com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException: AWS CloudFormation stack reached an error state: CREATE_FAILED reason: API: autoscaling:CreateAutoScalingGroup The default Service-Linked Role for Auto Scaling could not be created. com.amazonaws.services.identitymanagement.model.AmazonIdentityManagementException: User: arn:aws:sts::<account id>:assumed-role/CredentialRole/hadoop-provisioning is not authorized to perform: iam:CreateServiceLinkedRole on resource: arn:aws:iam::<account id>:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling (Service: AmazonIdentityManagement; Status Code: 403; Error Code: AccessDenied; Request ID: cc25dd31-1a50-11e9-bef1-a990dfdb8f39)


Can you please help?

4 REPLIES 4

Re: Cluster creation failed on Cloudbreak hosted on AWS

@Pushpak Nandi

According to the AWS documentation, this might be your issue:

"Amazon EC2 Auto Scaling creates the AWSServiceRoleForAutoScaling service-linked role for you the first time that you create an Auto Scaling group but do not specify a different service-linked role.

Make sure that you have enabled the IAM permissions that allow an IAM entity (such as a user, group, or role) to create the service-linked role. Otherwise, the automatic creation fails. For more information, see Service-Linked Role Permissions in the IAM User Guide or the information about required user permissions in this guide."

Hope this helps!

Re: Cluster creation failed on Cloudbreak hosted on AWS

New Contributor

@pdarvasi: Ok, this is what my cb-policy.json looks like:

{
"Effect": "Allow",
"Action": [
"iam:ListRolePolicies",
"iam:GetRolePolicy",
"iam:ListAttachedRolePolicies",
"iam:ListInstanceProfiles",
"iam:PutRolePolicy",
"iam:PassRole",
"iam:GetRole"
],
"Resource": [
"*"
]
},

{
"Effect": "Allow",
"Action": [
"iam:ListRolePolicies",
"iam:GetRolePolicy",
"iam:ListAttachedRolePolicies",
"iam:ListInstanceProfiles",
"iam:PutRolePolicy",
"iam:PassRole",
"iam:GetRole"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"autoscaling:CreateAutoScalingGroup",
"autoscaling:CreateLaunchConfiguration",
"autoscaling:DeleteAutoScalingGroup",
"autoscaling:DeleteLaunchConfiguration",
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeScalingActivities",
"autoscaling:DetachInstances",
"autoscaling:ResumeProcesses",
"autoscaling:SuspendProcesses",
"autoscaling:UpdateAutoScalingGroup"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"kms:ListKeys",
"kms:ListKeyPolicies",
"kms:ListAliases"
],
"Resource": "*"
}

========================================================

Are you saying, I also need to add the below?

{ "Effect": "Allow",
"Action": [
"ec2:*"
],
"Resource": "*"
}

Re: Cluster creation failed on Cloudbreak hosted on AWS

@Pushpak Nandi

No, according to the documentation, you should create an autoscaling group with an admin user with enough rights and ensure that "AWSServiceRoleForAutoScaling" role has been created automatically.

After this has succeeded your cluster create should proceed further.

Hope this helps!

Re: Cluster creation failed on Cloudbreak hosted on AWS

New Contributor

@pdarvasi, @Pushpak Nandi I too faced this problem, which I resolved by adding the permission iam:CreateServiceLinkedRole to the json described in this document.