Kudu Master Directories

[#1]

Hello,

We have just installed Kudu in our test environment, and are currently running CDH 5.13.1. Due to this being a small POC environment, we only have 2 tablet servers, and a single master, making it only usable for functional testing.

There were 4 requried configuration properties upon installation of Kudu with CDH 5.13, for whichthe following were configured:

Kudu Master WAL Directory:

/data/kudu/master_wal

Kudu Master Data Directories

/data/kudu/master_wal

Kudu Tablet Server WAL Directory

/data1/kudu/tablet_wal

Kudu Tablet Server Data Directories

/data1/kudu/tablet_data

/data2/kudu/tablet_data

/data3/kudu/tablet_data

My question is concerning the Master data directories configuration property:

Should multiple directories be used for storing the Kudu master data? It appears this is expected with the configuration property being plural, and it's set up to be configured similar to the tablet server data directories from Cloudera Manager. But if the Kudu Master server resides on one of the master/utility nodes, then there are not multiple JBOD mount points like a worker node.

Are there significant benefits of having multiple Kudu master data directories or inherit risks with just a single master data directory? If we configured an additional master data directory on the OS disk (such as under /var or /opt), would this be a concern?

I've read that SSDs are recommended for the WAL directories. Is there a major performance impact if the WAL directory is on the same mount point as one of the data directories?

Thank you,

Braz

[#2]

Thanks for these answers. Having issues with replying, but for your last answer concerning the WAL directory and the metadata, would you recommend having a separate directory for the Tablet Server WAL?

Thanks

[#3]

Yep, that would be ideal in that background flushes/compactions would not affect write performance and Raft elections.

[#4]

Okay, and I hope I'm not asking too much in this one forum post, but since it's related: what is the recommended number of Tablet Server directories?

Could we number of directories to each JBOD disk used by the DataNode? Of course without using a sub-directory of the DataNode.

[#5]

That is up to your workload and how much storage you need per node. It's common to see anywhere from 6 to 12 disks per tablet server. Check out the limitations documentation for some guidance there.