Member since
10-05-2018
6
Posts
0
Kudos Received
0
Solutions
11-15-2018
10:05 AM
ok thanks @pdarvasi
... View more
11-14-2018
07:11 PM
For upgrade testing, I've deployed HDP2.6.5.0-292 (w/ Ambari 2.6.2.2) from Cloubreak and followed the documentation to upgrade to Ambari 2.7.1.0. It fails during the "ambari-server upgrade " process # ambari-server upgrade
Using python /usr/bin/python
Upgrading ambari-server
INFO: Upgrade Ambari Server
INFO: Updating Ambari Server properties in ambari.properties ...
INFO: Updating Ambari Server properties in ambari-env.sh ...
INFO: Original file ambari-env.sh kept
WARNING: Original file krb5JAASLogin.conf kept
INFO: File krb5JAASLogin.conf updated.
INFO: Fixing database objects owner
Ambari Server configured for Postgres. Confirm you have made a backup of the Ambari Server database [y/n] (n)? y
INFO: Upgrading database schema
ERROR: Unexpected ValueError: No JSON object could be decoded
Traceback (most recent call last):
File "/usr/sbin/ambari-server.py", line 1060, in <module>
mainBody()
File "/usr/sbin/ambari-server.py", line 1030, in mainBody
main(options, args, parser)
File "/usr/sbin/ambari-server.py", line 980, in main
action_obj.execute()
File "/usr/sbin/ambari-server.py", line 79, in execute
self.fn(*self.args, **self.kwargs)
File "/usr/lib/ambari-server/lib/ambari_server/serverUpgrade.py", line 262, in upgrade
retcode = run_schema_upgrade(args)
File "/usr/lib/ambari-server/lib/ambari_server/serverUpgrade.py", line 162, in run_schema_upgrade
upgrade_response = json.loads(stdout)
File "/usr/lib/ambari-server/lib/ambari_simplejson/__init__.py", line 307, in loads
return _default_decoder.decode(s)
File "/usr/lib/ambari-server/lib/ambari_simplejson/decoder.py", line 335, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/ambari-server/lib/ambari_simplejson/decoder.py", line 353, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
... View more
Labels:
10-12-2018
04:09 PM
I found out that my Ambari server node to could communicate externaly due to /etc/resolv.conf # cat /etc/resolv.conf
; Created by cloud-init on instance boot automatically, do not edit. ;
; generated by /etc/dhcp/dhclient-enter-hooks
search openstacklocal nameserver 127.0.0.1 I added a valid DNS server and the HDP cluster was able to be deployed !
... View more
10-12-2018
03:40 PM
It does not work with keystone v2 credential as well.
Once I workaround the problem for downloading the image ( Q217819 ), it still fails with: Cannot use the specified Ambari stack: StackRepoDetails{stack='HDP-2.6'; utils='HDP-UTILS-1.1.0.22'}. Error: An internal system exception occurred: Could not load url from http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/HDP-2.6.5.0-292.xml. public-repo-1.hortonworks.com
... View more
10-05-2018
11:51 AM
Importing the image manually into Glance resolves the issue
... View more
10-05-2018
09:17 AM
I'm getting the same issue. I know Openstack Queen is not in the supported list of supported OS releases but I have no other choices for now. Cluster deployment fails with : cloudbreak_1 | 2018-10-04 14:05:26,663 [reactorDispatcher-76] handleImageNotFound:64 INFO c.s.c.c.o.c.OpenStackImageVerifier - [owner:2ec305f7-89aa-4f5b-b13b-e9afca6af487] [type:STACK] [id:3] [name:hdp-test] [flow:5c475919-a4e1-4e02-bd5e-932e336e93c6] [tracking:] Available images: Cirros-0.3.5, entry: GlanceImage{id=a22da222-76d2-421a-b5e4-bdf36007bfd9, name=Cirros-0.3.5, tags=[], imageStatus=ACTIVE, containerFormat=BARE, diskFormat=QCOW2, createdAt=Wed Sep 13 12:18:49 UTC 2017, updatedAt=Tue Mar 27 12:41:21 UTC 2018, minDisk=1, minRam=0, isProtected=true, checksum=f8ab98ff5e73ebab884d80c9dc9c7290, owner=17ed6d01ebee40ae89fa35f462bdd027, visibility=PUBLIC, size=13267968, locations=null, directUrl=file:///var/lib/glance/images/a22da222-76d2-421a-b5e4-bdf36007bfd9, self=/v2/images/a22da222-76d2-421a-b5e4-bdf36007bfd9, file=/v2/images/a22da222-76d2-421a-b5e4-bdf36007bfd9/file, schema=/v2/schemas/image, architecture=x86_64, instanceUuid=null, kernelId=null, osVersion=null, osDistro=null, ramdiskId=null, virtualSize=null}
cloudbreak_1 | 2018-10-04 14:05:26,664 [reactorDispatcher-76] handleImageNotFound:67 WARN c.s.c.c.o.c.OpenStackImageVerifier - [owner:2ec305f7-89aa-4f5b-b13b-e9afca6af487] [type:STACK] [id:3] [name:hdp-test] [flow:5c475919-a4e1-4e02-bd5e-932e336e93c6] [tracking:] OpenStack image: cb-hdp-26-1807091850 not found
cloudbreak_1 | 2018-10-04 14:05:26,665 [reactorDispatcher-76] importImage:35 INFO c.s.c.c.o.c.OpenStackImageImporter - [owner:2ec305f7-89aa-4f5b-b13b-e9afca6af487] [type:STACK] [id:3] [name:hdp-test] [flow:5c475919-a4e1-4e02-bd5e-932e336e93c6] [tracking:] Import OpenStack image from: http://public-repo-1.hortonworks.com/HDP/cloudbreak/cb-hdp-26-1807091850.img
cloudbreak_1 | 2018-10-04 14:05:26,665 [reactorDispatcher-76] isAccessible:33 INFO c.s.c.c.s.u.UrlAccessValidationService - [owner:2ec305f7-89aa-4f5b-b13b-e9afca6af487] [type:STACK] [id:3] [name:hdp-test] [flow:5c475919-a4e1-4e02-bd5e-932e336e93c6] [tracking:] Validation of url access: http://public-repo-1.hortonworks.com/HDP/cloudbreak/cb-hdp-26-1807091850.img
cloudbreak_1 | 2018-10-04 14:05:26,737 [reactorDispatcher-76] importImage:42 INFO c.s.c.c.o.c.OpenStackImageImporter - [owner:2ec305f7-89aa-4f5b-b13b-e9afca6af487] [type:STACK] [id:3] [name:hdp-test] [flow:5c475919-a4e1-4e02-bd5e-932e336e93c6] [tracking:] Executing of the following import Task: {image_properties={name=cb-hdp-26-1807091850, disk_format=qcow2, container_format=bare}, import_from=http://public-repo-1.hortonworks.com/HDP/cloudbreak/cb-hdp-26-1807091850.img, import_from_format=qcow2}
cloudbreak_1 | 2018-10-04 14:05:26,920 [reactorDispatcher-76] buildLogContextForReactorHandler:69 INFO c.s.c.l.LogContextAspects - [owner:2ec305f7-89aa-4f5b-b13b-e9afca6af487] [type:STACK] [id:3] [name:hdp-test] [flow:5c475919-a4e1-4e02-bd5e-932e336e93c6] [tracking:] A Reactor event handler's 'accept' method has been intercepted: execution(Flow2Handler.accept(..)), MDC logger context is built.
cloudbreak_1 | 2018-10-04 14:05:26,937 [reactorDispatcher-76] execute:87 INFO c.s.c.c.f.AbstractAction - [owner:2ec305f7-89aa-4f5b-b13b-e9afca6af487] [type:STACKVIEW] [id:3] [name:hdp-test] [flow:5c475919-a4e1-4e02-bd5e-932e336e93c6] [tracking:] Stack: 3, flow state: IMAGESETUP_STATE, phase: service, execution time 1 sec
cloudbreak_1 | 2018-10-04 14:05:26,938 [reactorDispatcher-76] handleStackCreationFailure:213 ERROR c.s.c.c.f.s.p.a.StackCreationService - [owner:2ec305f7-89aa-4f5b-b13b-e9afca6af487] [type:STACKVIEW] [id:3] [name:hdp-test] [flow:5c475919-a4e1-4e02-bd5e-932e336e93c6] [tracking:] Error during stack creation flow:
cloudbreak_1 | org.openstack4j.api.exceptions.ClientResponseException: Status: 403, Reason: Forbidden
cloudbreak_1 | at org.openstack4j.core.transport.HttpExceptionHandler.mapException(HttpExceptionHandler.java:38)
cloudbreak_1 | at org.openstack4j.core.transport.HttpExceptionHandler.mapException(HttpExceptionHandler.java:23)
cloudbreak_1 | at org.openstack4j.core.transport.HttpEntityHandler.handleLessThan500(HttpEntityHandler.java:101)
cloudbreak_1 | at org.openstack4j.core.transport.HttpEntityHandler.handle(HttpEntityHandler.java:47)
cloudbreak_1 | at org.openstack4j.core.transport.HttpEntityHandler.handle(HttpEntityHandler.java:24)
cloudbreak_1 | at org.openstack4j.connectors.jersey2.HttpResponseImpl.getEntity(HttpResponseImpl.java:63)
cloudbreak_1 | at org.openstack4j.openstack.internal.BaseOpenStackService$Invocation.execute(BaseOpenStackService.java:220)
cloudbreak_1 | at org.openstack4j.openstack.internal.BaseOpenStackService$Invocation.execute(BaseOpenStackService.java:202)
cloudbreak_1 | at org.openstack4j.openstack.image.v2.internal.TaskServiceImpl.create(TaskServiceImpl.java:45)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.openstack.common.OpenStackImageImporter.importImage(OpenStackImageImporter.java:43)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.openstack.common.OpenStackSetup.prepareImage(OpenStackSetup.java:51)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.PrepareImageHandler.accept(PrepareImageHandler.java:47)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.PrepareImageHandler.accept(PrepareImageHandler.java:21)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.PrepareImageHandler$FastClassBySpringCGLIB$aaf961cd.invoke(<generated>)
cloudbreak_1 | at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
cloudbreak_1 | at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
cloudbreak_1 | at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:52)
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
cloudbreak_1 | at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
cloudbreak_1 | at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.PrepareImageHandler$EnhancerBySpringCGLIB$cd32b61.accept(<generated>)
cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:317)
cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:310)
cloudbreak_1 | at reactor.bus.routing.ConsumerFilteringRouter.route(ConsumerFilteringRouter.java:72)
cloudbreak_1 | at reactor.bus.routing.TraceableDelegatingRouter.route(TraceableDelegatingRouter.java:51)
cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:591)
cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:63)
cloudbreak_1 | at reactor.core.dispatch.AbstractLifecycleDispatcher.route(AbstractLifecycleDispatcher.java:160)
cloudbreak_1 | at reactor.core.dispatch.MultiThreadDispatcher$MultiThreadTask.run(MultiThreadDispatcher.java:74)
cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
cloudbreak_1 | at java.lang.Thread.run(Thread.java:748)
cloudbreak_1 | 2018-10-04 14:05:26,994 [reactorDispatcher-76] fireCloudbreakEvent:58 INFO c.s.c.s.e.DefaultCloudbreakEventService - [owner:2ec305f7-89aa-4f5b-b13b-e9afca6af487] [type:STACKVIEW] [id:3] [name:hdp-test] [flow:5c475919-a4e1-4e02-bd5e-932e336e93c6] [tracking:] Firing Cloudbreak event: entityId: 3, type: CREATE_FAILED, message: Infrastructure creation failed. Reason: Status: 403, Reason: Forbidden I will try to import cb-hdp-26-1807091850.img manually in Glance as I'm able to download it via curl.
... View more
