Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

CDSW 1.6.1 fails to start

avatar
Explorer

After upgrading the Cloudera Manager from 6.2.0 to 6.3.1, the CDSW won't come online anymore. In the health logs, this line stands out:

 

MaxRetryError: HTTPSConnectionPool(host='xxx.xxx.xxx.xxx', port=6443): Max retries exceeded with url: /api/v1/secrets (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f364478d210>: Failed to establish a new connection: [Errno 111] Connection refused',))

 The port is closed on the CDSW master node, and I couldn't find any service in the docs that depends on it. Is it Kubernetes?

Further, from cdsw validate it seems that some chains are missing from iptables:

The following chains are missing from iptables: [KUBE-EXTERNAL-SERVICES, WEAVE-NPC-EGRESS, WEAVE-NPC, WEAVE-NPC-EGRESS-ACCEPT, KUBE-SERVICES, WEAVE-NPC-INGRESS, WEAVE-NPC-EGRESS-DEFAULT, WEAVE-NPC-DEFAULT, WEAVE-NPC-EGRESS-CUSTOM]

 However, I cannot remember any step in the installation process that required to set such rules, so I assume that was automated.

 

Is there a way to resolve the issue without rolling back the version?

1 ACCEPTED SOLUTION

avatar
Explorer

Was able to resolve this myself while encountering more and more familiar problems. These are the important take-aways:

  1. update the rest of CDH to same version as CM (6.3.1),
  2. enable ipv6 for all hosts,
  3. prepare the nodes using the CM,
  4. restart CDSW

Probably forgot a couple of steps here, but these are the ones I remember. At least, the goal of resolving the issue without a rollback was achieved.

View solution in original post

1 REPLY 1

avatar
Explorer

Was able to resolve this myself while encountering more and more familiar problems. These are the important take-aways:

  1. update the rest of CDH to same version as CM (6.3.1),
  2. enable ipv6 for all hosts,
  3. prepare the nodes using the CM,
  4. restart CDSW

Probably forgot a couple of steps here, but these are the ones I remember. At least, the goal of resolving the issue without a rollback was achieved.