Removing a Kudu Master fails

[#1]

Hello,

I am in the process in removing an obsolete Kudu master from a cluster, following the exact instructions and after starting everything I get this:

Configuration error: leader master liveness check error: Could not connect to the cluster: no leader master found. Client configured with 1 master(s) (node3.demo.lab:7051) but cluster indicates it expects 2 master(s) (node2.demo.lab:7051,node3.demo.lab:7051)

From the Kudu status page it says clearly that node3 is the leading master, and the only master on the cluster as desired.

All local config file have been cleared from node2, and the instructions I followed are these:

https://docs.cloudera.com/documentation/enterprise/6/6.2/topics/kudu_administration_cli.html#concept...

Any ideas where else to look?

Thanks!

J

[#2]

Poking around the code a bit for that error message, it seems like it might be coming from a missed step in the process. Did you perform step 1 here?

https://docs.cloudera.com/documentation/enterprise/6/6.2/topics/kudu_administration_cli.html#concept...

"Rewrite the Raft configuration on the remaining masters to include only the remaining masters. See Step 4 of Perform the Migration for more details."

If so, you should be able to confirm by running `sudo -u kudu kudu pbc dump /path/to/master/consensus-meta/00000000000000000000000000000000` and verifying that the committed config has only a single peer. Is that the case?