Support Questions
Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Advanced Search

CDSW 1.2.0 parcel installation is stuck in start after upgrade from 1.1.1 package installation

webtransactor
Explorer

Created ‎10-23-2017 02:14 AM

Hi, I tried to upgrade my 1.1.1 installation of CDSW to 1.2.0 parcel installation. But the Workbench gets stuck on startup in following state:

 

2017-10-23 11:11:26,408 INFO cdsw.status:OK: Application running as root check
2017-10-23 11:11:26,428 ERROR cdsw.status:Status check failed for services: [docker, kubelet, cdsw-app, cdsw-host-controller]
2017-10-23 11:11:26,451 INFO cdsw.status:OK: Sysctl params check
2017-10-23 11:11:26,461 INFO cdsw.status:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|         NAME         |   STATUS   |           CREATED-AT          |   VERSION   |   EXTERNAL-IP   |                     OS-IMAGE                    |       KERNEL-VERSION      |   GPU   |   STATEFUL   |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|   dn184.pf4h.local   |    True    |   2017-10-23 09:03:02+00:00   |   v1.6.11   |       None      |   Red Hat Enterprise Linux Server 7.2 (Maipo)   |   3.10.0-327.el7.x86_64   |    0    |     True     |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2017-10-23 11:11:26,462 INFO cdsw.status:1/1 nodes are ready.
2017-10-23 11:11:26,492 INFO cdsw.status:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|                     NAME                     |   READY   |    STATUS   |   RESTARTS   |           CREATED-AT          |        POD-IP       |       HOST-IP       |   ROLE   |
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|            etcd-dn184.pf4h.local             |    1/1    |   Running   |      0       |   2017-10-23 09:04:08+00:00   |   192.168.239.184   |   192.168.239.184   |   None   |
|       kube-apiserver-dn184.pf4h.local        |    1/1    |   Running   |      0       |   2017-10-23 09:04:05+00:00   |   192.168.239.184   |   192.168.239.184   |   None   |
|   kube-controller-manager-dn184.pf4h.local   |    1/1    |   Running   |      0       |   2017-10-23 09:03:58+00:00   |   192.168.239.184   |   192.168.239.184   |   None   |
|          kube-dns-3911048160-8klgz           |    3/3    |   Running   |      0       |   2017-10-23 09:03:15+00:00   |      100.66.0.4     |   192.168.239.184   |   None   |
|               kube-proxy-7bdr8               |    1/1    |   Running   |      0       |   2017-10-23 09:03:15+00:00   |   192.168.239.184   |   192.168.239.184   |   None   |
|       kube-scheduler-dn184.pf4h.local        |    1/1    |   Running   |      0       |   2017-10-23 09:03:49+00:00   |   192.168.239.184   |   192.168.239.184   |   None   |
|       node-problem-detector-v0.1-4g2j6       |    1/1    |   Running   |      0       |   2017-10-23 09:04:31+00:00   |   192.168.239.184   |   192.168.239.184   |   None   |
|               weave-net-70vj1                |    2/2    |   Running   |      0       |   2017-10-23 09:03:15+00:00   |   192.168.239.184   |   192.168.239.184   |   None   |
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2017-10-23 11:11:26,492 INFO cdsw.status:All required pods are ready in cluster kube-system.
2017-10-23 11:11:26,541 INFO cdsw.status:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|                  NAME                  |   READY   |     STATUS    |   RESTARTS   |           CREATED-AT          |        POD-IP       |       HOST-IP       |           ROLE           |
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|          cron-962987953-6wcdb          |    1/1    |    Running    |      0       |   2017-10-23 09:04:31+00:00   |      100.66.0.6     |   192.168.239.184   |           cron           |
|           db-875553086-jt2lr           |    1/1    |    Running    |      0       |   2017-10-23 09:04:31+00:00   |      100.66.0.8     |   192.168.239.184   |            db            |
|        db-migrate-d573dd7-twxxk        |    0/1    |   Succeeded   |      0       |   2017-10-23 09:04:31+00:00   |     100.66.0.10     |   192.168.239.184   |        db-migrate        |
|           engine-deps-t9c86            |    1/1    |    Running    |      0       |   2017-10-23 09:04:30+00:00   |      100.66.0.5     |   192.168.239.184   |       engine-deps        |
|   ingress-controller-506514573-k6tk7   |    1/1    |    Running    |      0       |   2017-10-23 09:04:30+00:00   |   192.168.239.184   |   192.168.239.184   |    ingress-controller    |
|        livelog-1589742313-rcpv1        |    1/1    |    Running    |      0       |   2017-10-23 09:04:31+00:00   |      100.66.0.7     |   192.168.239.184   |         livelog          |
|      reconciler-1584998901-z17nv       |    1/1    |    Running    |      0       |   2017-10-23 09:04:31+00:00   |      100.66.0.9     |   192.168.239.184   |        reconciler        |
|       spark-port-forwarder-tql0q       |    1/1    |    Running    |      0       |   2017-10-23 09:04:31+00:00   |   192.168.239.184   |   192.168.239.184   |   spark-port-forwarder   |
|           web-53233289-419hq           |    1/1    |    Running    |      0       |   2017-10-23 09:04:31+00:00   |     100.66.0.12     |   192.168.239.184   |           web            |
|           web-53233289-nhc60           |    1/1    |    Running    |      0       |   2017-10-23 09:04:31+00:00   |     100.66.0.13     |   192.168.239.184   |           web            |
|           web-53233289-sn3cx           |    1/1    |    Running    |      0       |   2017-10-23 09:04:31+00:00   |     100.66.0.11     |   192.168.239.184   |           web            |
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2017-10-23 11:11:26,542 INFO cdsw.status:All required pods are ready in cluster default.
2017-10-23 11:11:26,548 INFO cdsw.status:All required Application services are configured.
2017-10-23 11:11:26,553 INFO cdsw.status:All required config maps are ready.
2017-10-23 11:11:26,573 INFO cdsw.status:All required secrets are available.
2017-10-23 11:11:26,577 INFO cdsw.status:Persistent volumes are ready.
2017-10-23 11:11:26,580 INFO cdsw.status:Persistent volume claims are ready.
2017-10-23 11:11:26,585 INFO cdsw.status:Ingresses are ready.
2017-10-23 11:11:26,731 ERROR cdsw.status:Web is not yet up.
2017-10-23 11:11:26,731 INFO cdsw.monitor:{'metric': 'cdsw_status', 'timestampMs': 1508749886731, 'type': 'FAILURE', 'message': 'Web is not yet up.', 'entity': {'type': 'SERVICE', 'name': 'cdsw'}}
2017-10-23 11:11:26,731 INFO cdsw.monitor:Sending status to CM: {'statusRecords': [{'metric': 'cdsw_status', 'timestampMs': 1508749886731, 'type': 'FAILURE', 'message': 'Web is not yet up.', 'entity': {'type': 'SERVICE', 'name': 'cdsw'}}]}
2017-10-23 11:11:26,739 INFO cdsw.monitor:Successfully sent status to CM

But I can login to the Workbench. Only sessions cannot be startet, they all fail with status 1.

 

Reply
1,643 Views
0 Kudos
1 REPLY 1

Re: CDSW 1.2.0 parcel installation is stuck in start after upgrade from 1.1.1 package installation

webtransactor
Explorer

Created ‎10-23-2017 04:48 AM

After some configuration changes I found the reason: The proxy settings caused the problem. I started the cdsw without proxies and now it comes up and is working properly. I used the recommended NO_PROXY settings from the installation guide which worked with 1.1.1, but these don't work with 1.2.0. Is there an update for this setting?

Reply
1,631 Views
Don't have an account?
Register
Announcements
What's New @ Cloudera
Cloud Resource Validation tool (cdpctl) provides pre-validation of cloud resources, significantly reducing onboarding time
What's New @ Cloudera
Cloudera Machine Learning now enables administrators to register custom ML Runtimes
What's New @ Cloudera
Cloudera Machine Learning Runtimes are replacing Legacy Engines
What's New @ Cloudera
Cloudera Data Engineering (CDE) service diagnostic bundles improves service visibility and troubleshooting
What's New @ Cloudera
CDP Operational Database enables customers to use Spark with database through both SQL & NoSQL interfaces
View More Announcements