09-30-2018 11:24 PM - edited 10-01-2018 02:15 AM
Hello to all, I want to understand how kudu stores data on disk as log files, I've the following question:
1) Which property decides the how big a log file would be in kudu tserver data directory? After which a new log file is created. Is it
If not then which property? What should be the ideal size that I should keep?
2) Currently in my Production environment inside Tservers data directory, many 11GB log files exists, but upon checking documentation the size shown by ls -l is apprent size and to check the real size I should run du -h, but to my surprise all 11GB files don't have same real size, it ranges from 8KB to 600MB. Currently no compressing is enabled. So I wanted to understand if this behaviour is expected or unexpected and if yes why the real file size is different.
3) Accoring to what I understood that each tablet will have multi log files(because of reaching maximum log file size limit and after which it'll start appending to a new log file). Is my understanding correct? If yes, how can I know log files belongs to which tablet? currently I've close to 500 tablet, and more than 4K data files(log files). I wanted to understand the mapping of the files to a tablet.
10-01-2018 10:35 AM
10-02-2018 11:11 PM
First of all thanks for the reply.
Re Re1 - Okay currently --log_container_max_size is on default.
Re Re2 - So maximum size a log file in data directory can differ, what factor can cause that? What actually Log file contains, if WAL files have the real table data?
Re Re3 - Okay, so WAL files and tablets can be mapped? Then why number of WAL files and tablets differ? There are more WAL file than number of tablets.
My installed KUDU version is 1.6.0