Created 03-23-2019 08:08 PM
hi all
what is the preferred file system for data disks for kafka and data-node machines
XFS or EXT4 ?
note - for now we are using - ext4
Created 03-23-2019 08:19 PM
I do not think there is any specific recommendation for HDFS/Kafka in terms of XFS/EXT4 but from OS side there is a good comparison and description is available here:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/storage_administration...
.
Some generic filesystem recommendations from hwx docs: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_cluster-planning/content/ch_partitioning...
.
HCC Thread: https://community.hortonworks.com/questions/5086/ext4-vs-xfs-filesystem-survey-of-popularity.html
Created 03-23-2019 08:49 PM
@Jay - I see in the second link something that I not agree , link say - " Swap: Size 2X system memory " , as I know we need to disable the swap on all datanode and kafka machines !
Created 03-23-2019 08:59 PM
I feel you are right.
I also see similar threads on HCC for the Swap recommendations:
1. Your Old HCC discussion on Swap: https://community.hortonworks.com/questions/202215/swap-size-howrtonworks-recommendation-in-hadoop-c...
2. Other HCC discussion on Swap: https://community.hortonworks.com/questions/22548/what-is-the-hortonworks-recommendation-on-swap-usa...
Created 03-23-2019 09:08 PM
@Jay , since the swap is very critical , then who can update the hortonworks document ? , I think many people that saw that maybe do the mistake with the swap , so it must be update soon