Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Adding a master server in kudu

Solved Go to solution
Highlighted

Adding a master server in kudu

Expert Contributor

Hi,

 

I have installed kudu on my CDH 5.11.2 cluster with 2 masters. I want to add on emore master now. When i tried the regular way, my existing masters failed to start. Any thoughts are welcome.

 

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Adding a master server in kudu

Expert Contributor

Had to format the TS to get them started.

View solution in original post

7 REPLIES 7
Highlighted

Re: Adding a master server in kudu

Contributor

Hi Rajesh,

 

Would you please be more specific about what are the error log you saw? Thanks!

 

Best,

Hao

Highlighted

Re: Adding a master server in kudu

Expert Contributor

@Hao Hao

 

Error:

Unable to init master catalog manager: Invalid argument: Unable to initialize catalog manager: Failed to initialize sys tables async: on-disk and provided master lists are different.

Highlighted

Re: Adding a master server in kudu

Contributor
How did you add the master? Did you follow the steps outlined in the master
migration guide?

https://kudu.apache.org/docs/administration.html#migrate_to_multi_master
Highlighted

Re: Adding a master server in kudu

Expert Contributor

Hi @awong

 

I added it like anyother role in cloudera manager.

Highlighted

Re: Adding a master server in kudu

Expert Contributor

Hi @awong

 

I tried from the document you mentioned and have below exception while copying the data from existing masters.

 

kudu local_replica copy_from_remote --fs_wal_dir=/data/1/kudu/wal --fs_data_dirs=/data/1/kudu/data 00000000000000000000000000000000 xxxxxxxxxxx.migration.santanderuk.corp:7051
I0221 10:35:07.571077 37523 mem_tracker.cc:140] MemTracker: hard memory limit is 201.702087 GB
I0221 10:35:07.571144 37523 mem_tracker.cc:142] MemTracker: soft memory limit is 121.021248 GB
I0221 10:35:07.571871 37523 env_posix.cc:1308] Raising process file limit from 1024 to 4096
I0221 10:35:07.571907 37523 file_cache.cc:401] Constructed file cache lbm with capacity 1638
I0221 10:35:07.572870 37523 log_block_manager.cc:1343] Data dir /data/1/kudu/data/data is on an ext4 filesystem vulnerable to KUDU-1508 with block size 4096
I0221 10:35:07.572883 37523 log_block_manager.cc:1353] Limiting containers on data directory /data/1/kudu/data/data to 2721 blocks
I0221 10:35:07.573066 37523 fs_manager.cc:256] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I0221 10:35:07.573223 37523 fs_manager.cc:259] Opened local filesystem: /data/1/kudu/data,/data/1/kudu/wal
uuid: "32cd48071fbe40ef8b4c8f2726d52962"
format_stamp: "Formatted at 2018-02-21 09:59:54 on xxxxxxxx"
I0221 10:35:07.577749 37523 tablet_copy_client.cc:166] T 00000000000000000000000000000000 P 32cd48071fbe40ef8b4c8f2726d52962: Tablet Copy client: Beginning tablet copy session from remote peer at address srvcloprlsk332.migration.santanderuk.corp:7051
Remote error: Unable to begin tablet copy session: Not authorized: unauthorized access to method: BeginTabletCopySession: Invalid argument: Unable to decode tablet copy RPC error message: message: "Not authorized: unauthorized access to method: BeginTabletCopySession" code: ERROR_APPLICATION

 

I'm running this as root user. My cluster is kerborized.

Highlighted

Re: Adding a master server in kudu

Expert Contributor

I was able to sync the data after runnig the command as kudu user. But unfortunately, two tablet servers are not starting with below error.

 

 

Bad status: Already present: Could not create new FS layout: FSManager root is not empty: /data/7/kudu_ts/data

Re: Adding a master server in kudu

Expert Contributor

Had to format the TS to get them started.

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here