How do I get CloudBreak to create a cluster in AWS: image opt-in could not be validated

rt2357 — Fri, 02 Feb 2018 22:12:41 GMT

When trying to create a cluster on AWS, I get the following exceptions;

cloudbreak_1 | 2018-02-01 21:00:12,797 [http-nio-8080-exec-6] create:186 INFO c.s.c.s.c.AmbariClusterService - [owner:a290539e-7056-4492-8831-23d497654084] [type:springLog] [id:] [name:] Cluster requested [BlueprintId: 1]
cloudbreak_1 | 2018-02-01 21:00:13,011 [reactorDispatcher-9] validateImageOptIn:164 ERROR c.s.c.c.a.AwsSetup - [owner:a290539e-7056-4492-8831-23d497654084] [type:STACK] [id:7] [name:storm] Image opt-in could not be validated for AMI 'ami-7cb35904'.
cloudbreak_1 | com.amazonaws.AmazonClientException: Unrecognized service response for the dry-run request.
cloudbreak_1 | at com.amazonaws.services.ec2.AmazonEC2Client.dryRun(AmazonEC2Client.java:13557)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.aws.AwsSetup.validateImageOptIn(AwsSetup.java:148)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.aws.AwsSetup.prerequisites(AwsSetup.java:108)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.ProvisionSetupHandler.accept(ProvisionSetupHandler.java:50)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.ProvisionSetupHandler.accept(ProvisionSetupHandler.java:21)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.ProvisionSetupHandler$$FastClassBySpringCGLIB$$bbcf5b79.invoke(<generated>)
cloudbreak_1 | at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
cloudbreak_1 | at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
cloudbreak_1 | at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:52)
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
cloudbreak_1 | at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
cloudbreak_1 | at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.ProvisionSetupHandler$$EnhancerBySpringCGLIB$$ed30f4ac.accept(<generated>)
cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:317)
cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:310)
cloudbreak_1 | at reactor.bus.routing.ConsumerFilteringRouter.route(ConsumerFilteringRouter.java:72)
cloudbreak_1 | at reactor.bus.routing.TraceableDelegatingRouter.route(TraceableDelegatingRouter.java:51)
cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:591)
cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:63)
cloudbreak_1 | at reactor.core.dispatch.AbstractLifecycleDispatcher.route(AbstractLifecycleDispatcher.java:160)
cloudbreak_1 | at reactor.core.dispatch.MultiThreadDispatcher$MultiThreadTask.run(MultiThreadDispatcher.java:74)
cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
cloudbreak_1 | at java.lang.Thread.run(Thread.java:745)
cloudbreak_1 | Caused by: com.amazonaws.services.ec2.model.AmazonEC2Exception: No default VPC for this user (Service: AmazonEC2; Status Code: 400; Error Code: VPCIdNotSpecified; Request ID: e0fbafe7-f44e-4bc0-be32-5e31eda2b17a)
cloudbreak_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588)
cloudbreak_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258)
cloudbreak_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
cloudbreak_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
cloudbreak_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
cloudbreak_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
cloudbreak_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
cloudbreak_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
cloudbreak_1 | at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
cloudbreak_1 | at com.amazonaws.services.ec2.AmazonEC2Client.doInvoke(AmazonEC2Client.java:13611)
cloudbreak_1 | at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:13587)
cloudbreak_1 | at com.amazonaws.services.ec2.AmazonEC2Client.dryRun(AmazonEC2Client.java:13549)
cloudbreak_1 | ... 25 common frames omitted
cloudbreak_1 | 2018-02-01 21:00:13,056 [reactorDispatcher-9] accept:140 DEBUG c.s.c.c.f.Flow2Handler - [owner:a290539e-7056-4492-8831-23d497654084] [type:STACK] [id:7] [name:storm] flow control event arrived: key: SETUPRESULT_ERROR, flowid: 384a1d99-4eba-4e16-ba29-5e71534c852a, payload: CloudPlatformResult{status=FAILED, statusReason='You are not authorized to perform this operation.', errorDetails=com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException: You are not authorized to perform this operation., request=CloudStackRequest{, cloudStack=CloudStack{groups=[com.sequenceiq.cloudbreak.cloud.model.Group@c94202e, com.sequenceiq.cloudbreak.cloud.model.Group@6491d02d, com.sequenceiq.cloudbreak.cloud.model.Group@43807e4d, com.sequenceiq.cloudbreak.cloud.model.Group@4f5952db, com.sequenceiq.cloudbreak.cloud.model.Group@62e19e4e], network=com.sequenceiq.cloudbreak.cloud.model.Network@81b3a87, image=Image{imageName='ami-7cb35904', userdata={CORE=#!/bin/bash
cloudbreak_1 |
cloudbreak_1 | ## logging
cloudbreak_1 | exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
cloudbreak_1 |
cloudbreak_1 | set -x
cloudbreak_1 |
cloudbreak_1 | export CLOUD_PLATFORM="AWS"
cloudbreak_1 | export START_LABEL=97
cloudbreak_1 | export PLATFORM_DISK_PREFIX=xvd
cloudbreak_1 | export LAZY_FORMAT_DISK_LIMIT=12
cloudbreak_1 | export IS_GATEWAY=false
cloudbreak_1 | export TMP_SSH_KEY="ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCbuDfdt6ukwFNintr+cEQGY/X0sKSnXIsW1jh1OOdJYUMM89fCgpRbF1ixUEPo1xAau6R+iRVt9fx9tavGc1nYnj3F/CxRkK2uYrlfDzBsDXTDJUqvcGddF4KgkJzof4u/nQEhQnsJE5RQWqj9lOrAxscIDXD3WysLN6x09ih+lgtg+sK1/W37RqLLA+ECEpFIUq5k/8QadS+8pCR6ITenoF0b62LSBcDod64bOuWddRtBaQnY7XkL9lGWESiPA/Mj+pQzpignmuNijnXvH+UHvH9zikK02/Ee0E6ySveE4MDAtFc+1AbIf5benDsZoJQuwD1/MCR4ZYmSBaYdX9H9 cloudbreak
cloudbreak_1 | "
cloudbreak_1 | export PUBLIC_SSH_KEY="ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCRxh8AoPplqmlo0fP2F2uTyh073gKvLARehGwtetY7JOtn2SDf1h5M7q3Ilq4/Nl9KGoTRzMtXlVPixDsldImLtuwipS0+BAjjpdsXIeTSCmSYWS6DUh2PG/UP0gdX8TIFBhElCqhGnTPKhhXCmXH0GFlx/Z3rTTK6khsahEtH9R+tvBdPa70D6yLbVFW578UBb8TzfsAmbcRlPojn79CaW6/LYP8I+1dxo/7MGU31yJb2sF4YEMiEdA0Mu5k8rogyUqlJbpcAHJLW4y+elBW7A8KUHF+3RcWsLBiFUBt+22mC2TR8PaDomjlLEmO07vlF61Zh1LeIWiIVg1D9Ee4d rt2357@att.com"
cloudbreak_1 | export RELOCATE_DOCKER=false
cloudbreak_1 | export SSH_USER=cloudbreak
cloudbreak_1 | export SALT_BOOT_PASSWORD=2cq7kg6360sk0e6qrq4o1nk9dp
cloudbreak_1 | export SALT_BOOT_SIGN_KEY=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAm7g33berpMBTYp7a/nBEBmP19LCkp1yLFtY4dTjnSWFDDPPXwoKUWxdYsVBD6NcQGrukfokVbfX8fbWrxnNZ2J49xfwsUZCtrmK5Xw8wbA10wyVKr3BnXReCoJCc6H+Lv50BIUJ7CROUUFqo/ZTqwMbHCA1w91srCzesdPYofpYLYPrCtf1t+0aiywPhAhKRSFKuZP/EGnUvvKQkeiE3p6BdG+ti0gXA6HeuGzrlnXUbQWkJ2O15C/ZRlhEojwPzI/qUM6YoJ5rjYo517x/lB7x/c4pCtNvxHtBOskr3hODAwLRXPtQGyH+W3pw7GaCULsA9fzAkeGWJkgWmHV/R/QIDAQAB
cloudbreak_1 |
cloudbreak_1 | touch /tmp/cb-custom-data-default.txt
cloudbreak_1 |
cloudbreak_1 | /usr/bin/user-data-helper.sh "$@" &> /var/log/user-data.log
cloudbreak_1 | , GATEWAY=#!/bin/bash
cloudbreak_1 |
cloudbreak_1 | ## logging
cloudbreak_1 | exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
cloudbreak_1 |
cloudbreak_1 | set -x
cloudbreak_1 |
cloudbreak_1 | export CLOUD_PLATFORM="AWS"
cloudbreak_1 | export START_LABEL=97
cloudbreak_1 | export PLATFORM_DISK_PREFIX=xvd
cloudbreak_1 | export LAZY_FORMAT_DISK_LIMIT=12
cloudbreak_1 | export IS_GATEWAY=true
cloudbreak_1 | export TMP_SSH_KEY="ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCbuDfdt6ukwFNintr+cEQGY/X0sKSnXIsW1jh1OOdJYUMM89fCgpRbF1ixUEPo1xAau6R+iRVt9fx9tavGc1nYnj3F/CxRkK2uYrlfDzBsDXTDJUqvcGddF4KgkJzof4u/nQEhQnsJE5RQWqj9lOrAxscIDXD3WysLN6x09ih+lgtg+sK1/W37RqLLA+ECEpFIUq5k/8QadS+8pCR6ITenoF0b62LSBcDod64bOuWddRtBaQnY7XkL9lGWESiPA/Mj+pQzpignmuNijnXvH+UHvH9zikK02/Ee0E6ySveE4MDAtFc+1AbIf5benDsZoJQuwD1/MCR4ZYmSBaYdX9H9 cloudbreak
cloudbreak_1 | "
cloudbreak_1 | export PUBLIC_SSH_KEY="ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCRxh8AoPplqmlo0fP2F2uTyh073gKvLARehGwtetY7JOtn2SDf1h5M7q3Ilq4/Nl9KGoTRzMtXlVPixDsldImLtuwipS0+BAjjpdsXIeTSCmSYWS6DUh2PG/UP0gdX8TIFBhElCqhGnTPKhhXCmXH0GFlx/Z3rTTK6khsahEtH9R+tvBdPa70D6yLbVFW578UBb8TzfsAmbcRlPojn79CaW6/LYP8I+1dxo/7MGU31yJb2sF4YEMiEdA0Mu5k8rogyUqlJbpcAHJLW4y+elBW7A8KUHF+3RcWsLBiFUBt+22mC2TR8PaDomjlLEmO07vlF61Zh1LeIWiIVg1D9Ee4d rt2357@att.com"
cloudbreak_1 | export RELOCATE_DOCKER=false
cloudbreak_1 | export SSH_USER=cloudbreak
cloudbreak_1 | export SALT_BOOT_PASSWORD=2cq7kg6360sk0e6qrq4o1nk9dp
cloudbreak_1 | export SALT_BOOT_SIGN_KEY=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAm7g33berpMBTYp7a/nBEBmP19LCkp1yLFtY4dTjnSWFDDPPXwoKUWxdYsVBD6NcQGrukfokVbfX8fbWrxnNZ2J49xfwsUZCtrmK5Xw8wbA10wyVKr3BnXReCoJCc6H+Lv50BIUJ7CROUUFqo/ZTqwMbHCA1w91srCzesdPYofpYLYPrCtf1t+0aiywPhAhKRSFKuZP/EGnUvvKQkeiE3p6BdG+ti0gXA6HeuGzrlnXUbQWkJ2O15C/ZRlhEojwPzI/qUM6YoJ5rjYo517x/lB7x/c4pCtNvxHtBOskr3hODAwLRXPtQGyH+W3pw7GaCULsA9fzAkeGWJkgWmHV/R/QIDAQAB
cloudbreak_1 |
cloudbreak_1 | touch /tmp/cb-custom-data-default.txt
cloudbreak_1 |
cloudbreak_1 | /usr/bin/user-data-helper.sh "$@" &> /var/log/user-data.log
cloudbreak_1 | }}}}}
cloudbreak_1 | 2018-02-01 21:00:13,124 [reactorDispatcher-9] execute:73 INFO c.s.c.c.f.AbstractAction - [owner:a290539e-7056-4492-8831-23d497654084] [type:STACK] [id:7] [name:storm] Stack: 7, flow state: SETUP_STATE, phase: service, execution time 0 sec
cloudbreak_1 | 2018-02-01 21:00:13,125 [reactorDispatcher-9] handleStackCreationFailure:231 ERROR c.s.c.c.f.s.p.a.StackCreationService - [owner:a290539e-7056-4492-8831-23d497654084] [type:STACK] [id:7] [name:storm] Error during stack creation flow:
cloudbreak_1 | com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException: You are not authorized to perform this operation.
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.aws.AwsSetup.prerequisites(AwsSetup.java:118)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.ProvisionSetupHandler.accept(ProvisionSetupHandler.java:50)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.ProvisionSetupHandler.accept(ProvisionSetupHandler.java:21)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.ProvisionSetupHandler$$FastClassBySpringCGLIB$$bbcf5b79.invoke(<generated>)
cloudbreak_1 | at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
cloudbreak_1 | at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
cloudbreak_1 | at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:52)
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
cloudbreak_1 | at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
cloudbreak_1 | at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.ProvisionSetupHandler$$EnhancerBySpringCGLIB$$ed30f4ac.accept(<generated>)
cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:317)
cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:310)
cloudbreak_1 | at reactor.bus.routing.ConsumerFilteringRouter.route(ConsumerFilteringRouter.java:72)
cloudbreak_1 | at reactor.bus.routing.TraceableDelegatingRouter.route(TraceableDelegatingRouter.java:51)
cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:591)
cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:63)
cloudbreak_1 | at reactor.core.dispatch.AbstractLifecycleDispatcher.route(AbstractLifecycleDispatcher.java:160)
cloudbreak_1 | at reactor.core.dispatch.MultiThreadDispatcher$MultiThreadTask.run(MultiThreadDispatcher.java:74)
cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
cloudbreak_1 | at java.lang.Thread.run(Thread.java:745)
cloudbreak_1 | 2018-02-01 21:00:13,125 [reactorDispatcher-9] fireEventAndLog:25 DEBUG c.s.c.c.f.s.FlowMessageService - [owner:a290539e-7056-4492-8831-23d497654084] [type:STACK] [id:7] [name:storm] STACK_INFRASTRUCTURE_CREATE_FAILED [STACK_FLOW_STEP].

Re: How do I get CloudBreak to create a cluster in AWS: image opt-in could not be validated

anarasimham — Tue, 06 Feb 2018 04:36:38 GMT

@Bob Thorman

According to your stack trace, you may not have the requisite permissions to perform this operation. Please check your AWS user permissions and make sure you have enough capacity to allocate the cluster you are trying to allocate.

	cloudbreak_1 | 2018-02-01 21:00:13,056 [reactorDispatcher-9] accept:140 
DEBUG c.s.c.c.f.Flow2Handler - 
[owner:a290539e-7056-4492-8831-23d497654084] [type:STACK] [id:7] 
[name:storm] flow control event arrived: key: SETUPRESULT_ERROR, flowid:
 384a1d99-4eba-4e16-ba29-5e71534c852a, payload: 
CloudPlatformResult{status=FAILED, statusReason='You are not authorized 
to perform this operation.', 
errorDetails=com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException:
 You are not authorized to perform this operation., 
request=CloudStackRequest{, 
cloudStack=CloudStack{groups=[com.sequenceiq.cloudbreak.cloud.model.Group@c94202e,
 com.sequenceiq.cloudbreak.cloud.model.Group@6491d02d, 
com.sequenceiq.cloudbreak.cloud.model.Group@43807e4d, 
com.sequenceiq.cloudbreak.cloud.model.Group@4f5952db, 
com.sequenceiq.cloudbreak.cloud.model.Group@62e19e4e],

Re: How do I get CloudBreak to create a cluster in AWS: image opt-in could not be validated

rt2357 — Tue, 06 Feb 2018 07:26:34 GMT

I'm an admin in our AWS account. I don't think it was my permissions regarding resource allocation. I can create EC2 instances of the same profile manually. I think its an issue with me using the HDP AMI. Are there some special HDP polices I needs for my role?

Re: How do I get CloudBreak to create a cluster in AWS: image opt-in could not be validated

anarasimham — Wed, 07 Feb 2018 03:23:15 GMT

No, shouldn't be any special HDP policies. Perhaps you are running up against an upper quota? Cores/RAM/disk set by either AWS or your organization?

Re: How do I get CloudBreak to create a cluster in AWS: image opt-in could not be validated

darvasip — Fri, 09 Feb 2018 20:48:11 GMT

@Bob Thorman

As we have already spoken, the root cause was a bug in Cloudbreak when no default VPC was present in the AWS account.

The solution is to include the following line in his Profile file and do a "cbd restart" afterwards.

export DOCKER_TAG_CLOUDBREAK=1165-patch

Re: How do I get CloudBreak to create a cluster in AWS: image opt-in could not be validated

rt2357 — Fri, 09 Feb 2018 21:08:07 GMT

Yes, thank you @pdarvasi that was the ticket. I appreciate all your help.

question Re: How do I get CloudBreak to create a cluster in AWS: image opt-in could not be validated in Archives of Support Questions (Read Only)

How do I get CloudBreak to create a cluster in AWS: image opt-in could not be validated

Re: How do I get CloudBreak to create a cluster in AWS: image opt-in could not be validated

Re: How do I get CloudBreak to create a cluster in AWS: image opt-in could not be validated

Re: How do I get CloudBreak to create a cluster in AWS: image opt-in could not be validated

Re: How do I get CloudBreak to create a cluster in AWS: image opt-in could not be validated

Re: How do I get CloudBreak to create a cluster in AWS: image opt-in could not be validated