Created 03-22-2017 03:06 PM
Hey everyone,
I am currently trying to deploy an Hadoop Cluster using Cloudbreak 1.6.3 with the blueprint hdp-small-default over Openstack Ocata. I'm working behind a Proxy.
I got an issue : Infrastructure creation failed. Reason: null
I don't use HEAT so i'm using the NATIVE option to deploy HDP Cluster.
Any idea how could I fix that ?
Thank you very much!
Created 03-23-2017 09:37 AM
It turned out that you have found an issue, so I am filing a Jira ticket for that.
As a workaround, you might try to get it working with HEAT (or even with Keystone V2), it should work.
Hope this helps!
Created 03-22-2017 09:40 PM
Please note, that Ocata is not supported officially by Cloudbreak yet, you can check the documentation.
It might work but afaik it was not tested against it. It would be the most useful if you could send the Cloudbreak logs (either from your cbd directory or generated by "cbd logs cloudbreak"), it should contain the root cause of the error.
Hope this helps!
Created 03-23-2017 08:59 AM
tank you very much for your answer. I have seen this documentation so I can understand that some problems happened ...
I guess that the problem occurs just after credentials creation but I don't understand very well the reason ...
I give you logs details :
cloudbreak_1 | 2017-03-23 08:43:40,379 [reactorDispatcher-36] buildLogContextForReactorHandler:48 INFO c.s.c.l.LogContextAspects - [owner:1013d790-f4ad-4fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] A Reactor event handler's 'accept' method has been inter cepted: execution(Consumer.accept(..)), MDC logger context is built. cloudbreak_1 | 2017-03-23 08:43:40,379 [reactorDispatcher-36] accept:56 INFO c.s.c.c.h.LaunchStackHandler - [owner:1013d790-f4ad-4 fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] Received event: Event{id=null, headers={FLOW_CHAIN_ID=561f3d40-9e74-4512-a8ee-0 90a6c444efe, FLOW_ID=4590e3de-d876-45d3-bed7-ea9bb5d83942}, replyTo=null, key=LAUNCHSTACKREQUEST, data=CloudStackRequest{, cloudStack=CloudS tack{groups=[com.sequenceiq.cloudbreak.cloud.model.Group@626305be, com.sequenceiq.cloudbreak.cloud.model.Group@6542b76a, com.sequenceiq.clou dbreak.cloud.model.Group@79237cb3, com.sequenceiq.cloudbreak.cloud.model.Group@5f42ace4, com.sequenceiq.cloudbreak.cloud.model.Group@25f61c4 7], network=com.sequenceiq.cloudbreak.cloud.model.Network@557f1674, image=com.sequenceiq.cloudbreak.cloud.model.Image@375b2e8c}}} cloudbreak_1 | 2017-03-23 08:43:40,379 [reactorDispatcher-36] authenticate:37 INFO c.s.c.c.o.a.OpenStackAuthenticator - [owner:101 3d790-f4ad-4fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] Authenticating to openstack ... cloudbreak_1 | 2017-03-23 08:43:40,703 [reactorDispatcher-36] create:45 INFO c.s.c.c.o.a.OpenStackCredentialConnector - [owner:101 3d790-f4ad-4fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] Create credential: com.sequenceiq.cloudbreak.cloud.model.CloudCrede ntial@7d34810f cloudbreak_1 | 2017-03-23 08:43:41,105 [reactorDispatcher-36] create:58 INFO c.s.c.c.o.a.OpenStackCredentialConnector - [owner:101 3d790-f4ad-4fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] Credential has been created: com.sequenceiq.cloudbreak.cloud.model. CloudCredential@7d34810f, kp: NovaKeypair{name=cb-testluter_10-cluser-2, public_key=ssh-rsa AAAAB****Hl2Eq0b Generated-by-Nova, fingerprint=08:a3:d5:1a:ea:59:17:e9:17:85:ba:a5:e6:da:c5:97} cloudbreak_1 | 2017-03-23 08:43:41,106 [reactorDispatcher-36] accept:70 DEBUG c.s.c.c.f.Flow2Handler - [owner:1013d790-f4ad-4fe3-b1 04-dbea54b31e71] [type:STACK] [id:10] [name:testluter] flow control event arrived: key: LAUNCHSTACKRESULT_ERROR, flowid: 4590e3de-d876-45d3- bed7-ea9bb5d83942, payload: LaunchStackResult{status=FAILED, statusReason='', errorDetails=java.lang.NullPointerException, request=CloudStac kRequest{, cloudStack=CloudStack{groups=[com.sequenceiq.cloudbreak.cloud.model.Group@626305be, com.sequenceiq.cloudbreak.cloud.model.Group@6 542b76a, com.sequenceiq.cloudbreak.cloud.model.Group@79237cb3, com.sequenceiq.cloudbreak.cloud.model.Group@5f42ace4, com.sequenceiq.cloudbre ak.cloud.model.Group@25f61c47], network=com.sequenceiq.cloudbreak.cloud.model.Network@557f1674, image=com.sequenceiq.cloudbreak.cloud.model. Image@375b2e8c}}, results=null} cloudbreak_1 | 2017-03-23 08:43:41,159 [reactorDispatcher-36] execute:64 INFO c.s.c.c.f.AbstractAction - [owner:1013d790-f4ad-4fe3 -b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] Stack: 10, flow state: START_PROVISIONING_STATE, phase: service, execution time 0 sec cloudbreak_1 | 2017-03-23 08:43:41,159 [reactorDispatcher-36] handleStackCreationFailure:176 ERROR c.s.c.c.f.s.p.a.StackCreationSer vice - [owner:1013d790-f4ad-4fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] Error during stack creation flow: cloudbreak_1 | java.lang.NullPointerException: null cloudbreak_1 | at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:210) cloudbreak_1 | at org.openstack4j.openstack.identity.internal.TenantServiceImpl.getByName(TenantServiceImpl.java:34) cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.openstack.nativ.context.OpenStackContextBuilder.contextInit(OpenStackCont extBuilder.java:46) cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.openstack.nativ.context.OpenStackContextBuilder.contextInit(OpenStackCont extBuilder.java:27) cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.template.AbstractResourceConnector.launch(AbstractResourceConnector.java: 55) cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.LaunchStackHandler.accept(LaunchStackHandler.java:69) cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.LaunchStackHandler.accept(LaunchStackHandler.java:33) cloudbreak_1 | at sun.reflect.GeneratedMethodAccessor237.invoke(Unknown Source) cloudbreak_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) cloudbreak_1 | at java.lang.reflect.Method.invoke(Method.java:498) cloudbreak_1 | at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.j ava:190) cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) cloudbreak_1 | at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceIntercep tor.java:52) cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) cloudbreak_1 | at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:9 2) cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) cloudbreak_1 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) cloudbreak_1 | at com.sun.proxy.$Proxy209.accept(Unknown Source) cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:317) cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:310) cloudbreak_1 | at reactor.bus.routing.ConsumerFilteringRouter.route(ConsumerFilteringRouter.java:72) cloudbreak_1 | at reactor.bus.routing.TraceableDelegatingRouter.route(TraceableDelegatingRouter.java:51) cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:591) cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:63) cloudbreak_1 | at reactor.core.dispatch.AbstractLifecycleDispatcher.route(AbstractLifecycleDispatcher.java:160) cloudbreak_1 | at reactor.core.dispatch.MultiThreadDispatcher$MultiThreadTask.run(MultiThreadDispatcher.java:74) cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) cloudbreak_1 | at java.lang.Thread.run(Thread.java:745) cloudbreak_1 | 2017-03-23 08:43:41,160 [reactorDispatcher-36] fireEventAndLog:24 DEBUG c.s.c.c.f.s.FlowMessageService - [owner:1013 d790-f4ad-4fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] STACK_INFRASTRUCTURE_CREATE_FAILED [STACK_FLOW_STEP]. cloudbreak_1 | 2017-03-23 08:43:41,160 [reactorDispatcher-36] fireCloudbreakEvent:51 INFO c.s.c.s.e.DefaultCloudbreakEventService - [owner:1013d790-f4ad-4fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] Firing Cloudbreak event: CloudbreakEventData{entityId=1 0, eventType='UPDATE_IN_PROGRESS', eventMessage='Infrastructure creation failed. Reason: null'} cloudbreak_1 | 2017-03-23 08:43:41,160 [reactorDispatcher-36] handleFailure:212 DEBUG c.s.c.c.f.s.p.a.StackCreationService - [owner :1013d790-f4ad-4fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] Nothing to do. OnFailureAction DO_NOTHING cloudbreak_1 | 2017-03-23 08:43:41,161 [reactorDispatcher-46] accept:33 INFO c.s.c.s.e.CloudbreakEventHandler - [owner:undefined] [type:CLOUDBREAKEVENTDATA] [id:undefined] [name:undefined] Handling cloudbreak event: Event{id=null, headers=null, replyTo=null, key=CLOUDBR EAK_EVENT, data=CloudbreakEventData{entityId=10, eventType='UPDATE_IN_PROGRESS', eventMessage='Infrastructure creation failed. Reason: null' }} cloudbreak_1 | 2017-03-23 08:43:41,161 [reactorDispatcher-46] accept:36 INFO c.s.c.s.e.CloudbreakEventHandler - [owner:undefined] [type:CLOUDBREAKEVENTDATA] [id:undefined] [name:undefined] Persisting data: CloudbreakEventData{entityId=10, eventType='UPDATE_IN_PROGRESS', eventMessage='Infrastructure creation failed. Reason: null'} cloudbreak_1 | 2017-03-23 08:43:41,162 [reactorDispatcher-46] createStackEvent:66 DEBUG c.s.c.s.e.DefaultCloudbreakEventService - [ owner:undefined] [type:CLOUDBREAKEVENTDATA] [id:undefined] [name:undefined] Creating stack event from: CloudbreakEventData{entityId=10, even tType='UPDATE_IN_PROGRESS', eventMessage='Infrastructure creation failed. Reason: null'} cloudbreak_1 | 2017-03-23 08:43:41,207 [reactorDispatcher-46] createStackEvent:71 INFO c.s.c.s.e.DefaultCloudbreakEventService - [ owner:undefined] [type:CLOUDBREAKEVENTDATA] [id:undefined] [name:undefined] Created stack event: CloudbreakEvent{id=50, eventType='UPDATE_IN _PROGRESS', eventTimestamp=Thu Mar 23 08:43:41 UTC 2017, eventMessage='Infrastructure creation failed. Reason: null', owner='1013d790-f4ad-4 fe3-b104-dbea54b31e71', account='seq1234567', cloud='OPENSTACK', region='RegionOne', blueprintName='hdp-small-default', instanceGroup='null' , blueprintId=1', stackId=10', clusterId=6', clusterName=testluter} cloudbreak_1 | 2017-03-23 08:43:41,208 [reactorDispatcher-46] accept:38 INFO c.s.c.s.e.CloudbreakEventHandler - [owner:undefined] [type:CLOUDBREAKEVENTDATA] [id:undefined] [name:undefined] Sending notification with data: CloudbreakEvent{id=50, eventType='UPDATE_IN_PROGR ESS', eventTimestamp=Thu Mar 23 08:43:41 UTC 2017, eventMessage='Infrastructure creation failed. Reason: null', owner='1013d790-f4ad-4fe3-b1 04-dbea54b31e71', account='seq1234567', cloud='OPENSTACK', region='RegionOne', blueprintName='hdp-small-default', instanceGroup='null', blue printId=1', stackId=10', clusterId=6', clusterName=testluter} cloudbreak_1 | 2017-03-23 08:43:41,241 [reactorDispatcher-36] fireEventAndLog:24 DEBUG c.s.c.c.f.s.FlowMessageService - [owner:1013 d790-f4ad-4fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] STACK_INFRASTRUCTURE_CREATE_FAILED [STACK_FLOW_STEP]. cloudbreak_1 | 2017-03-23 08:43:41,241 [reactorDispatcher-36] fireCloudbreakEvent:51 INFO c.s.c.s.e.DefaultCloudbreakEventService - [owner:1013d790-f4ad-4fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] Firing Cloudbreak event: CloudbreakEventData{entityId=1 0, eventType='CREATE_FAILED', eventMessage='Infrastructure creation failed. Reason: null'} cloudbreak_1 | 2017-03-23 08:43:41,241 [reactorDispatcher-36] sendEvent:103 INFO c.s.c.c.f.AbstractAction - [owner:1013d790-f4ad-4 fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] Triggering event: com.sequenceiq.cloudbreak.reactor.api.event.StackEvent@10ca94 6f cloudbreak_1 | 2017-03-23 08:43:41,241 [reactorDispatcher-36] stateChanged:125 INFO c.s.c.c.f.c.AbstractFlowConfiguration - [owner :1013d790-f4ad-4fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] changed from ObjectState [getIds()=[START_PROVISIONING_STATE], getClass()=class org.springframework.statemachine.state.ObjectState, hashCode()=1563669019, toString()=AbstractState [id=START_PROVISIONING _STATE, pseudoState=null, deferred=null, entryActions=[com.sequenceiq.cloudbreak.core.flow2.stack.provision.action.StackCreationActions$3@37 257c66], exitActions=null, regions=[], submachine=null]] to ObjectState [getIds()=[STACK_CREATION_FAILED_STATE], getClass()=class org.spring framework.statemachine.state.ObjectState, hashCode()=1418289361, toString()=AbstractState [id=STACK_CREATION_FAILED_STATE, pseudoState=null, deferred=null, entryActions=[com.sequenceiq.cloudbreak.core.flow2.stack.provision.action.StackCreationActions$8@227aa096], exitActions=null , regions=[], submachine=null]] cloudbreak_1 | 2017-03-23 08:43:41,242 [reactorDispatcher-36] accept:33 INFO c.s.c.s.e.CloudbreakEventHandler - [owner:1013d790-f4 ad-4fe3-b104-dbea54b31e71] [type:STACK] [id:10] [name:testluter] Handling cloudbreak event: Event{id=null, headers=null, replyTo=null, key=C LOUDBREAK_EVENT, data=CloudbreakEventData{entityId=10, eventType='CREATE_FAILED', eventMessage='Infrastructure creation failed. Reason: null '}} cloudbreak_1 | 2017-03-23 08:43:41,242 [reactorDispatcher-36] accept:36 INFO c.s.c.s.e.CloudbreakEventHandler - [owner:undefined] [type:CLOUDBREAKEVENTDATA] [id:undefined] [name:undefined] Persisting data: CloudbreakEventData{entityId=10, eventType='CREATE_FAILED', even tMessage='Infrastructure creation failed. Reason: null'} cloudbreak_1 | 2017-03-23 08:43:41,242 [reactorDispatcher-36] createStackEvent:66 DEBUG c.s.c.s.e.DefaultCloudbreakEventService - [ owner:undefined] [type:CLOUDBREAKEVENTDATA] [id:undefined] [name:undefined] Creating stack event from: CloudbreakEventData{entityId=10, even tType='CREATE_FAILED', eventMessage='Infrastructure creation failed. Reason: null'} cloudbreak_1 | 2017-03-23 08:43:41,242 [reactorDispatcher-45] accept:70 DEBUG c.s.c.c.f.Flow2Handler - [owner:1013d790-f4ad-4fe3-b1 04-dbea54b31e71] [type:STACK] [id:7] [name:clusteressai2] flow control event arrived: key: STACK_CREATION_FAILHANDLED, flowid: 4590e3de-d876 -45d3-bed7-ea9bb5d83942, payload: com.sequenceiq.cloudbreak.reactor.api.event.StackEvent@10ca946f cloudbreak_1 | 2017-03-23 08:43:41,262 [reactorDispatcher-45] execute:64 INFO c.s.c.c.f.AbstractAction - [owner:1013d790-f4ad-4fe3 -b104-dbea54b31e71] [type:STACK] [id:7] [name:clusteressai2] Stack: 10, flow state: STACK_CREATION_FAILED_STATE, phase: service, execution t ime 0 sec cloudbreak_1 | 2017-03-23 08:43:41,262 [reactorDispatcher-45] sendEvent:103 INFO c.s.c.c.f.AbstractAction - [owner:1013d790-f4ad-4 fe3-b104-dbea54b31e71] [type:STACK] [id:7] [name:clusteressai2] Triggering event: com.sequenceiq.cloudbreak.reactor.api.event.StackEvent@10c a946f cloudbreak_1 | 2017-03-23 08:43:41,262 [reactorDispatcher-45] stateChanged:125 INFO c.s.c.c.f.c.AbstractFlowConfiguration - [owner :1013d790-f4ad-4fe3-b104-dbea54b31e71] [type:STACK] [id:7] [name:clusteressai2] changed from ObjectState [getIds()=[STACK_CREATION_FAILED_S TATE], getClass()=class org.springframework.statemachine.state.ObjectState, hashCode()=1418289361, toString()=AbstractState [id=STACK_CREATI ON_FAILED_STATE, pseudoState=null, deferred=null, entryActions=[com.sequenceiq.cloudbreak.core.flow2.stack.provision.action.StackCreationAct ions$8@227aa096], exitActions=null, regions=[], submachine=null]] to ObjectState [getIds()=[FINAL_STATE], getClass()=class org.springframewo rk.statemachine.state.ObjectState, hashCode()=158514087, toString()=AbstractState [id=FINAL_STATE, pseudoState=org.springframework.statemach ine.state.DefaultPseudoState@7bcf0eef, deferred=null, entryActions=[com.sequenceiq.cloudbreak.core.flow2.FlowFinalizeAction@7ad046de], exitA ctions=null, regions=[], submachine=null]] cloudbreak_1 | 2017-03-23 08:43:41,263 [reactorDispatcher-45] stop:127 INFO o.s.s.s.LifecycleObjectSupport - [owner:1013d790-f4ad- 4fe3-b104-dbea54b31e71] [type:STACK] [id:7] [name:clusteressai2] stopped org.springframework.statemachine.support.DefaultStateMachineExecuto r@3717503e cloudbreak_1 | 2017-03-23 08:43:41,263 [reactorDispatcher-45] stop:127 INFO o.s.s.s.LifecycleObjectSupport - [owner:1013d790-f4ad- 4fe3-b104-dbea54b31e71] [type:STACK] [id:7] [name:clusteressai2] stopped STACK_CREATION_FINISHED_STATE START_PROVISIONING_STATE INIT_STATE S TACK_CREATION_FAILED_STATE TLS_SETUP_STATE PROVISIONING_FINISHED_STATE FINAL_STATE IMAGE_CHECK_STATE IMAGESETUP_STATE COLLECTMETADATA_STATE SETUP_STATE / / id=3d058f95-50d0-41b2-b1eb-5d07c6e7961b cloudbreak_1 | 2017-03-23 08:43:41,263 [reactorDispatcher-45] finalizeFlow:95 DEBUG c.s.c.c.f.Flow2Handler - [owner:1013d790-f4ad-4 fe3-b104-dbea54b31e71] [type:STACK] [id:7] [name:clusteressai2] flow finalizing arrived: id: 4590e3de-d876-45d3-bed7-ea9bb5d83942 cloudbreak_1 | 2017-03-23 08:43:41,281 [reactorDispatcher-36] createStackEvent:71 INFO c.s.c.s.e.DefaultCloudbreakEventService - [ owner:undefined] [type:CLOUDBREAKEVENTDATA] [id:undefined] [name:undefined] Created stack event: CloudbreakEvent{id=51, eventType='CREATE_FA ILED', eventTimestamp=Thu Mar 23 08:43:41 UTC 2017, eventMessage='Infrastructure creation failed. Reason: null', owner='1013d790-f4ad-4fe3-b 104-dbea54b31e71', account='seq1234567', cloud='OPENSTACK', region='RegionOne', blueprintName='hdp-small-default', instanceGroup='null', blu eprintId=1', stackId=10', clusterId=6', clusterName=testluter} cloudbreak_1 | 2017-03-23 08:43:41,281 [reactorDispatcher-36] accept:38 INFO c.s.c.s.e.CloudbreakEventHandler - [owner:undefined] [type:CLOUDBREAKEVENTDATA] [id:undefined] [name:undefined] Sending notification with data: CloudbreakEvent{id=51, eventType='CREATE_FAILED', eventTimestamp=Thu Mar 23 08:43:41 UTC 2017, eventMessage='Infrastructure creation failed. Reason: null', owner='1013d790-f4ad-4fe3-b104-db ea54b31e71', account='seq1234567', cloud='OPENSTACK', region='RegionOne', blueprintName='hdp-small-default', instanceGroup='null', blueprint Id=1', stackId=10', clusterId=6', clusterName=testluter}
Created 03-23-2017 09:37 AM
It turned out that you have found an issue, so I am filing a Jira ticket for that.
As a workaround, you might try to get it working with HEAT (or even with Keystone V2), it should work.
Hope this helps!