question Re: Can Cloudbreak deploy HDP on custom AMI's? in Support Questions

Can Cloudbreak deploy HDP on custom AMI's?

mjrhee — Sun, 24 Jul 2016 19:01:12 GMT

Hi,

I would like Cloudbreak to use a custom AMI for my HDP nodes on AWS, but there doesn't seem to be a way to do this in the UI or shell. Is there a way to do this using the API? Or some other potential route? What kinds of pre-configuration are required?

Best,

Michael

Re: Can Cloudbreak deploy HDP on custom AMI's?

rdoktorics — Mon, 25 Jul 2016 20:09:23 GMT

Hi @mjrhee,

Cloudbreak shell now part of the cloudbreak repository here: https://github.com/sequenceiq/cloudbreak/tree/master/shell and this repository https://github.com/sequenceiq/cloudbreak-shell is no longer supported (also the --image tag) as the README mention. Currently you can define custom image with this process: http://sequenceiq.com/cloudbreak-docs/master/images/ If you need any other information just let me know.

Br,

Richard

Re: Can Cloudbreak deploy HDP on custom AMI's?

mjrhee — Mon, 25 Jul 2016 23:18:42 GMT

Hi @rdoktorics,

Thanks for the tip. I have tried creating and editing the etc/aws-images.yml file as the docs describe, stopped and restarted Cloudbreak, but it's still defaulting to the same AMI as before (us-west-1:ami-3a750e5a).

From what I can tell, the AMI is pulled into the code when the application is built. See here: https://github.com/sequenceiq/cloudbreak/blob/46bc01c25300460a275ef08121d6f4a566ec0db0/cloud-aws/aws-images.tmpl and https://github.com/sequenceiq/cloudbreak/blob/46bc01c25300460a275ef08121d6f4a566ec0db0/cloud-aws/src/main/resources/aws-images.yml

In my case, Cloudbreak always uses the AMI defined in version 55 of this atlas: https://atlas.hashicorp.com/api/v1/artifacts/sequenceiq/cloudbreak/amazon.image/search

FYI, I am using the pre-built cloud image of Cloudbreak 1.3 in us-west-1 on AWS. Is there something else I'm missing?

Best, Michael

*EDIT: Added Cloudbreak version for clarity.

Re: Can Cloudbreak deploy HDP on custom AMI's?

rdoktorics — Tue, 26 Jul 2016 16:00:26 GMT

Could you please add this line to your Profile: export CBD_FORCE_START=true

and also please write this into the compose file where the cloudbreak container is declared under the volume definition:

- ./etc/:/etc/cloudbreak

after the modification the volume section will seems like this:

volumes:

- "/Users/rdoktorics/Projects/cloudbreak-deployer/certs:/certs"

- /dev/urandom:/dev/random

- ./etc/:/etc/cloudbreak

after that please do not regenerate the compose file. This is just a patch because cbd 1.3.0 does not implement this feature. The future releases will contains this fix.

If you have any problem applying the patch then I am happy to help on webex.

Br,

Richard

Re: Can Cloudbreak deploy HDP on custom AMI's?

mjrhee — Wed, 27 Jul 2016 02:18:06 GMT

This worked on v1.3. Thank you so much, @rdoktorics!

I also was able to use the modified aws-images.yml file on the latest build of master without changing any Profile or config, so that can be an option for us to consider as well.

Also, is there any documentation for how to pre-configure an AMI for Cloudbreak to be able to deploy nodes successfully? Does it handle the standard HDP pre-config steps at all? We'd like to use RHEL, if possible.

Thanks again!

Best,

Michael

Re: Can Cloudbreak deploy HDP on custom AMI's?

rdoktorics — Thu, 28 Jul 2016 15:14:03 GMT

You should contact with the Hortonworks support and they will help you to create the custom AMI's.

Br,

Richard

Re: Can Cloudbreak deploy HDP on custom AMI's?

wendell_bu — Tue, 06 Sep 2016 03:38:17 GMT

Hi,

I followed the instructions, and added my ami in the etc/aws-images.yml.

I did see the HDP/Ambari VMs were created from the image. But I got timeout error when cloudbreak ssh into the VM.

Error logs from cloudbreak

/cbreak_cloudbreak_1 | 2016-09-05 18:02:54,899 [reactorDispatcher-37] checkStatus:31 ESC[34mINFO ESC[0;39m c.s.c.s.s.f.SshCheckerTask - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Connecting with ssh
to: 52.8.62.29, user: ec2-user, privatekey: /certs/stack-3/cb-ssh-key-3
/cbreak_cloudbreak_1 | 2016-09-05 18:02:54,941 [reactorDispatcher-37] checkStatus:37 ESC[34mINFO ESC[0;39m c.s.c.s.s.f.SshCheckerTask - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Failed to connect ss
h: Exhausted available authentication methods
/cbreak_cloudbreak_1 | 2016-09-05 18:02:54,941 [reactorDispatcher-37] notifyDisconnect:75 ESC[34mINFO ESC[0;39m n.s.s.t.TransportImpl - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Disconnected - BY_AP
PLICATION
/cbreak_cloudbreak_1 | 2016-09-05 18:02:59,961 [reactorDispatcher-37] pollWithTimeout:30 ESC[34mINFO ESC[0;39m c.s.c.s.PollingService - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Polling attempt 99.
/cbreak_cloudbreak_1 | 2016-09-05 18:02:59,963 [reactorDispatcher-37] init:155 ESC[34mINFO ESC[0;39m n.s.s.t.TransportImpl - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Client identity string: SSH-2.0
-SSHJ_0_14_0
/cbreak_cloudbreak_1 | 2016-09-05 18:02:59,968 [reactorDispatcher-37] init:168 ESC[34mINFO ESC[0;39m n.s.s.t.TransportImpl - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Server identity string: SSH-2.0
-OpenSSH_5.3

/cbreak_cloudbreak_1 | 2016-09-05 18:03:00,032 [reactorDispatcher-37] checkStatus:31 ESC[34mINFO ESC[0;39m c.s.c.s.s.f.SshCheckerTask - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Connecting with ssh
to: 52.8.62.29, user: ec2-user, privatekey: /certs/stack-3/cb-ssh-key-3
/cbreak_cloudbreak_1 | 2016-09-05 18:03:00,075 [reactorDispatcher-37] checkStatus:37 ESC[34mINFO ESC[0;39m c.s.c.s.s.f.SshCheckerTask - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Failed to connect ss
h: Exhausted available authentication methods
/cbreak_cloudbreak_1 | 2016-09-05 18:03:00,075 [reactorDispatcher-37] notifyDisconnect:75 ESC[34mINFO ESC[0;39m n.s.s.t.TransportImpl - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Disconnected - BY_AP
PLICATION
/cbreak_cloudbreak_1 | 2016-09-05 18:03:05,095 [reactorDispatcher-37] pollWithTimeout:55 ESC[34mINFO ESC[0;39m c.s.c.s.PollingService - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Poller timeout.
/cbreak_cloudbreak_1 | 2016-09-05 18:03:05,096 [reactorDispatcher-37] execute:73 ESC[1;31mERRORESC[0;39m c.s.c.c.f.AbstractAction - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Error during execution o
f com.sequenceiq.cloudbreak.core.flow2.stack.provision.action.StackCreationActions$6
/cbreak_cloudbreak_1 | com.sequenceiq.cloudbreak.service.CloudbreakServiceException: Operation timed out. Could not reach ssh connection in time
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.service.stack.flow.SshCheckerTask.handleTimeout(SshCheckerTask.java:50)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.service.stack.flow.SshCheckerTask.handleTimeout(SshCheckerTask.java:14)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.service.PollingService.pollWithTimeout(PollingService.java:56)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.service.PollingService.pollWithTimeoutSingleFailure(PollingService.java:66)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.service.stack.flow.TlsSetupService.waitForSsh(TlsSetupService.java:103)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.service.stack.flow.TlsSetupService.setupTls(TlsSetupService.java:83)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.stack.provision.action.StackCreationService.setupTls(StackCreationService.java:163)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.stack.provision.action.StackCreationActions$6.doExecute(StackCreationActions.java:160)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.stack.provision.action.StackCreationActions$6.doExecute(StackCreationActions.java:157)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.AbstractAction.execute(AbstractAction.java:70)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.state.ObjectState.entry(ObjectState.java:135)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.AbstractStateMachine.entryToState(AbstractStateMachine.java:860)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.AbstractStateMachine.setCurrentState(AbstractStateMachine.java:704)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.AbstractStateMachine.switchToState(AbstractStateMachine.java:628)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.AbstractStateMachine.access$200(AbstractStateMachine.java:80)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.AbstractStateMachine$2.transit(AbstractStateMachine.java:274)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.DefaultStateMachineExecutor.handleTriggerTrans(DefaultStateMachineExecutor.java:204)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.DefaultStateMachineExecutor.processTriggerQueue(DefaultStateMachineExecutor.java:342)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.DefaultStateMachineExecutor.access$100(DefaultStateMachineExecutor.java:57)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.DefaultStateMachineExecutor$1.run(DefaultStateMachineExecutor.java:231)
/cbreak_cloudbreak_1 |  at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.DefaultStateMachineExecutor.scheduleEventQueueProcessing(DefaultStateMachineExecutor.java:243)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.DefaultStateMachineExecutor.execute(DefaultStateMachineExecutor.java:141)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.AbstractStateMachine.sendEvent(AbstractStateMachine.java:201)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.FlowAdapter.sendEvent(FlowAdapter.java:54)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.Flow2Handler.accept(Flow2Handler.java:74)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.Flow2Handler.accept(Flow2Handler.java:23)
/cbreak_cloudbreak_1 |  at reactor.bus.EventBus$3.accept(EventBus.java:317)
/cbreak_cloudbreak_1 |  at reactor.bus.EventBus$3.accept(EventBus.java:310)
/cbreak_cloudbreak_1 |  at reactor.bus.routing.ConsumerFilteringRouter.route(ConsumerFilteringRouter.java:72)
/cbreak_cloudbreak_1 |  at reactor.bus.routing.TraceableDelegatingRouter.route(TraceableDelegatingRouter.java:51)
/cbreak_cloudbreak_1 |  at reactor.bus.EventBus.accept(EventBus.java:591)
/cbreak_cloudbreak_1 |  at reactor.bus.EventBus.accept(EventBus.java:63)
/cbreak_cloudbreak_1 |  at reactor.core.dispatch.AbstractLifecycleDispatcher.route(AbstractLifecycleDispatcher.java:160)
/cbreak_cloudbreak_1 |  at reactor.core.dispatch.MultiThreadDispatcher$MultiThreadTask.run(MultiThreadDispatcher.java:74)
/cbreak_cloudbreak_1 |  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
/cbreak_cloudbreak_1 |  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
/cbreak_cloudbreak_1 |  at java.lang.Thread.run(Thread.java:745)
/cbreak_cloudbreak_1 | 2016-09-05 18:03:05,096 [reactorDispatcher-37] sendEvent:103 ESC[34mINFO ESC[0;39m c.s.c.c.f.AbstractAction - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Triggering event: com.sequenceiq.cloudbreak.reactor.api.event.StackFailureEvent@64fde9f1

If I use the default ami, everything is ok. My cloudbreak version is 1.5.0-rc.9-371eb32

I used packer to build my image from official CentOS6, and I did set the ec2-user as default ssh user.

Not sure what special setup need in the ami?

Regards,

Wendell

Hi,

I followed the instructions, added the etc/aws-images.yml with my ami.

I did see the image is used and deployed as HDP/Ambari VMs. But I got the time out error when cloudbreak ssh into the VM.

/cbreak_cloudbreak_1 | 2016-09-05 18:02:54,899 [reactorDispatcher-37] checkStatus:31 ESC[34mINFO ESC[0;39m c.s.c.s.s.f.SshCheckerTask - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Connecting with ssh
to: 52.8.62.29, user: ec2-user, privatekey: /certs/stack-3/cb-ssh-key-3
/cbreak_cloudbreak_1 | 2016-09-05 18:02:54,941 [reactorDispatcher-37] checkStatus:37 ESC[34mINFO ESC[0;39m c.s.c.s.s.f.SshCheckerTask - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Failed to connect ss
h: Exhausted available authentication methods
/cbreak_cloudbreak_1 | 2016-09-05 18:02:54,941 [reactorDispatcher-37] notifyDisconnect:75 ESC[34mINFO ESC[0;39m n.s.s.t.TransportImpl - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Disconnected - BY_AP
PLICATION
/cbreak_cloudbreak_1 | 2016-09-05 18:02:59,961 [reactorDispatcher-37] pollWithTimeout:30 ESC[34mINFO ESC[0;39m c.s.c.s.PollingService - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Polling attempt 99.
/cbreak_cloudbreak_1 | 2016-09-05 18:02:59,963 [reactorDispatcher-37] init:155 ESC[34mINFO ESC[0;39m n.s.s.t.TransportImpl - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Client identity string: SSH-2.0
-SSHJ_0_14_0
/cbreak_cloudbreak_1 | 2016-09-05 18:02:59,968 [reactorDispatcher-37] init:168 ESC[34mINFO ESC[0;39m n.s.s.t.TransportImpl - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Server identity string: SSH-2.0
-OpenSSH_5.3
/cbreak_cloudbreak_1 | 2016-09-05 18:03:00,032 [reactorDispatcher-37] checkStatus:31 ESC[34mINFO ESC[0;39m c.s.c.s.s.f.SshCheckerTask - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Connecting with ssh
to: 52.8.62.29, user: ec2-user, privatekey: /certs/stack-3/cb-ssh-key-3
/cbreak_cloudbreak_1 | 2016-09-05 18:03:00,075 [reactorDispatcher-37] checkStatus:37 ESC[34mINFO ESC[0;39m c.s.c.s.s.f.SshCheckerTask - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Failed to connect ss
h: Exhausted available authentication methods
/cbreak_cloudbreak_1 | 2016-09-05 18:03:00,075 [reactorDispatcher-37] notifyDisconnect:75 ESC[34mINFO ESC[0;39m n.s.s.t.TransportImpl - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Disconnected - BY_AP
PLICATION

/cbreak_cloudbreak_1 | 2016-09-05 18:03:05,095 [reactorDispatcher-37] pollWithTimeout:55 ESC[34mINFO ESC[0;39m c.s.c.s.PollingService - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Poller timeout.
/cbreak_cloudbreak_1 | 2016-09-05 18:03:05,096 [reactorDispatcher-37] execute:73 ESC[1;31mERRORESC[0;39m c.s.c.c.f.AbstractAction - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Error during execution o
f com.sequenceiq.cloudbreak.core.flow2.stack.provision.action.StackCreationActions$6
/cbreak_cloudbreak_1 | com.sequenceiq.cloudbreak.service.CloudbreakServiceException: Operation timed out. Could not reach ssh connection in time
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.service.stack.flow.SshCheckerTask.handleTimeout(SshCheckerTask.java:50)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.service.stack.flow.SshCheckerTask.handleTimeout(SshCheckerTask.java:14)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.service.PollingService.pollWithTimeout(PollingService.java:56)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.service.PollingService.pollWithTimeoutSingleFailure(PollingService.java:66)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.service.stack.flow.TlsSetupService.waitForSsh(TlsSetupService.java:103)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.service.stack.flow.TlsSetupService.setupTls(TlsSetupService.java:83)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.stack.provision.action.StackCreationService.setupTls(StackCreationService.java:163)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.stack.provision.action.StackCreationActions$6.doExecute(StackCreationActions.java:160)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.stack.provision.action.StackCreationActions$6.doExecute(StackCreationActions.java:157)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.AbstractAction.execute(AbstractAction.java:70)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.state.ObjectState.entry(ObjectState.java:135)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.AbstractStateMachine.entryToState(AbstractStateMachine.java:860)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.AbstractStateMachine.setCurrentState(AbstractStateMachine.java:704)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.AbstractStateMachine.switchToState(AbstractStateMachine.java:628)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.AbstractStateMachine.access$200(AbstractStateMachine.java:80)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.AbstractStateMachine$2.transit(AbstractStateMachine.java:274)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.DefaultStateMachineExecutor.handleTriggerTrans(DefaultStateMachineExecutor.java:204)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.DefaultStateMachineExecutor.processTriggerQueue(DefaultStateMachineExecutor.java:342)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.DefaultStateMachineExecutor.access$100(DefaultStateMachineExecutor.java:57)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.DefaultStateMachineExecutor$1.run(DefaultStateMachineExecutor.java:231)
/cbreak_cloudbreak_1 |  at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.DefaultStateMachineExecutor.scheduleEventQueueProcessing(DefaultStateMachineExecutor.java:243)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.DefaultStateMachineExecutor.execute(DefaultStateMachineExecutor.java:141)
/cbreak_cloudbreak_1 |  at org.springframework.statemachine.support.AbstractStateMachine.sendEvent(AbstractStateMachine.java:201)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.FlowAdapter.sendEvent(FlowAdapter.java:54)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.Flow2Handler.accept(Flow2Handler.java:74)
/cbreak_cloudbreak_1 |  at com.sequenceiq.cloudbreak.core.flow2.Flow2Handler.accept(Flow2Handler.java:23)
/cbreak_cloudbreak_1 |  at reactor.bus.EventBus$3.accept(EventBus.java:317)
/cbreak_cloudbreak_1 |  at reactor.bus.EventBus$3.accept(EventBus.java:310)
/cbreak_cloudbreak_1 |  at reactor.bus.routing.ConsumerFilteringRouter.route(ConsumerFilteringRouter.java:72)
/cbreak_cloudbreak_1 |  at reactor.bus.routing.TraceableDelegatingRouter.route(TraceableDelegatingRouter.java:51)
/cbreak_cloudbreak_1 |  at reactor.bus.EventBus.accept(EventBus.java:591)
/cbreak_cloudbreak_1 |  at reactor.bus.EventBus.accept(EventBus.java:63)
/cbreak_cloudbreak_1 |  at reactor.core.dispatch.AbstractLifecycleDispatcher.route(AbstractLifecycleDispatcher.java:160)
/cbreak_cloudbreak_1 |  at reactor.core.dispatch.MultiThreadDispatcher$MultiThreadTask.run(MultiThreadDispatcher.java:74)
/cbreak_cloudbreak_1 |  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
/cbreak_cloudbreak_1 |  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
/cbreak_cloudbreak_1 |  at java.lang.Thread.run(Thread.java:745)
/cbreak_cloudbreak_1 | 2016-09-05 18:03:05,096 [reactorDispatcher-37] sendEvent:103 ESC[34mINFO ESC[0;39m c.s.c.c.f.AbstractAction - [owner:d1293e7e-a6fd-450c-bfb7-f42fa305a295] [type:STACK] [id:3] [name:spark2] Triggering event: com.sequenceiq.cloudbreak.reactor.api.event.StackFailureEvent@64fde9f1

But if I use the default ami, everything is ok. My cloudbreak version is 1.5.0-rc.9-371eb32

I used packer build the ami from official CentOS6, and do set the default ssh user as ec2-user.

So not sure what special setup need to be done in the ami?

Cheers,

Wendell

Re: Can Cloudbreak deploy HDP on custom AMI's?

karri_linnoinen — Wed, 28 Dec 2016 23:53:22 GMT

With Cloudbreak 1.6.1, you can use Custom Cloud Images, which you can specify in the /etc/ directory, as mentioned in this document: http://sequenceiq.com/cloudbreak-docs/latest/images/. There's no need to modify the Profile file.

If you add a custom AMI for AWS for example, remember to run cbd restart before deploying a cluster.

Cheers, K