Skip to main content
mfulwiler
mfulwiler
Silver ⭐️⭐️
April 3, 2019
Solved

Migrate one of three kudu master to a new server

Hi all, 

 

We are going through the hassle of rebuilding our boxes with CentOS7 as there is no upgrade path from CentOS6. Now I have looked over these Kudu documentation to migrating to multiple kudu master but the problem is the first warning it says it is unsafe to do this so I guess my question would be can I take one master making it a 2 node master then follow the documentation on migrating to a 3 node master setup. what process would I follow to remove the 3rd master other than just deleteing the role as I can only imagine how kudu will handle this. Also we are running kudu 1.4

 

Hao Hao
Hao HaoAnswer
Gold ⭐️
April 3, 2019

The following step below should help you:

  1. take a phyciscal backup of the master you are going to take down (in case something is going wrong with the migration).
  2. stop doing any DDL to the masters to prevent data loss. 
  3. take down the master you chosen on step 1.
  4. follow the doc Recovering from a Dead Kudu Master in a Multi-Master Deployment.
View in thread

3 replies

Hao Hao
Gold ⭐️
Hao HaoAnswer
Gold ⭐️
April 3, 2019

The following step below should help you:

  1. take a phyciscal backup of the master you are going to take down (in case something is going wrong with the migration).
  2. stop doing any DDL to the masters to prevent data loss. 
  3. take down the master you chosen on step 1.
  4. follow the doc Recovering from a Dead Kudu Master in a Multi-Master Deployment.
Translate
    mfulwiler
    mfulwilerAuthor
    Silver ⭐️⭐️
    April 4, 2019

    Thanks Hoa Hoa, for the reply I will test these steps out they seem pretty straight forwards thanks for you help. One thing I have a question about is step 2 you are stay to stop doing any DDL to the masters does this mean I should take down the other masters or make sure we dont have any DDL stametns running which may be a little tricky once we move to production. 

    Translate
    Hao Hao
    Gold ⭐️
    Hao Hao
    Gold ⭐️
    April 4, 2019

    You don't need to take down the other masters, but need to establish a maintenance window (no DDLs) to ensure the table metadata is consistent between followers and the leader. If it is hard in production, alternative is to ensure the master to be taken down is not the leader (to avoid lose data).

    Translate
    Terms & ConditionsAccessibility statement