what is the preferred file system for data disks for kafka and data-node machines
XFS or EXT4 ?
note - for now we are using - ext4
I do not think there is any specific recommendation for HDFS/Kafka in terms of XFS/EXT4 but from OS side there is a good comparison and description is available here:
Some generic filesystem recommendations from hwx docs: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_cluster-planning/content/ch_partitioning...
@Jay - I see in the second link something that I not agree , link say - " Swap: Size 2X system memory " , as I know we need to disable the swap on all datanode and kafka machines !
I feel you are right.
I also see similar threads on HCC for the Swap recommendations:
1. Your Old HCC discussion on Swap: https://community.hortonworks.com/questions/202215/swap-size-howrtonworks-recommendation-in-hadoop-c...
2. Other HCC discussion on Swap: https://community.hortonworks.com/questions/22548/what-is-the-hortonworks-recommendation-on-swap-usa...
@Jay , since the swap is very critical , then who can update the hortonworks document ? , I think many people that saw that maybe do the mistake with the swap , so it must be update soon