question Re: Ext4 vs XFS Filesystem - Survey of Popularity in Support Questions

Ext4 vs XFS Filesystem - Survey of Popularity

wfloyd — Thu, 03 Dec 2015 05:15:22 GMT

Wanted to see how many people have clusters where the HDFS DataNodes are running on XFS vs Ext4 filesystems? I'm trying to get a sense for which filesystem is chosen most often in the wild. Feel free to comment if you have a preference for one vs the other.

Thanks!

Re: Ext4 vs XFS Filesystem - Survey of Popularity

nsabharwal — Thu, 03 Dec 2015 05:16:34 GMT

@Wes Floyd All ext4 ...None XFS

Re: Ext4 vs XFS Filesystem - Survey of Popularity

bsaini — Thu, 03 Dec 2015 05:32:01 GMT

I've seen very little ext3 and mostly ext4 for the on prem deployments.

AWS EBS is xfs by default. XFS has its advantages but in a JBOD setup, it doesn't really provide lot of benefits.

Re: Ext4 vs XFS Filesystem - Survey of Popularity

jstraub — Thu, 03 Dec 2015 05:37:02 GMT

+1 .......

Re: Ext4 vs XFS Filesystem - Survey of Popularity

pcodding — Fri, 04 Dec 2015 06:42:40 GMT

From a SmartSense perspective it's about 80% ext4, and 20% XFS. We recommend using either and have specific mount options for each type of filesystem.

Re: Ext4 vs XFS Filesystem - Survey of Popularity

skumpf — Fri, 04 Dec 2015 07:02:00 GMT

FWIW, XFS is the default in RHEL 7, so I expect an uptick in new clusters.

Re: Ext4 vs XFS Filesystem - Survey of Popularity

nsabharwal — Fri, 04 Dec 2015 08:53:12 GMT

@Shane Kumpf @Wes Floyd

That's good to know https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Migration_Planning_Guide/sect-Red_Hat_Enterprise_Linux-Migration_Planning_Guide-File_System_Formats.html

XFS is a very high performance, scalable file system and is routinely deployed in the most demanding applications. In Red Hat Enterprise Linux 7, XFS is the default file system and is supported on all architectures. Ext4, which does not scale to the same size as XFS, is fully supported on all architectures and will continue to see active development and support.

Re: Ext4 vs XFS Filesystem - Survey of Popularity

bsaini — Fri, 04 Dec 2015 12:57:30 GMT

This should be updated / corrected then?

Partitioning Recommendations for Slave Nodes

Hadoop Slave node partitions: Hadoop should have its own partitions for Hadoop files and logs. Drives should be partitioned using ext3, ext4, or XFS, in that order of preference. HDFS on ext3 has been publicly tested on the Yahoo cluster, which makes it the safest choice for the underlying file system. The ext4 file system may have potential data loss issues with default options because of the "delayed writes" feature. XFS reportedly also has some data loss issues upon power failure. Do not use LVM; it adds latency and causes a bottleneck.

Source: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_cluster-planning-guide/content/ch_partitioning_chapter.html

A lot of this conflicts with the reality (Paul's Smartsense statistics) and what we all are discussing here.

Re: Ext4 vs XFS Filesystem - Survey of Popularity

dorio — Thu, 10 Dec 2015 05:50:08 GMT

We use all XFS but, during some benchmarks on the jobs, we changed to EXT4 for better performance (1-3% depending on the workflow).

Re: Ext4 vs XFS Filesystem - Survey of Popularity

nsabharwal — Thu, 10 Dec 2015 09:51:13 GMT

@Andrea D'Orio

Is it possible to publish benchmark numbers ?

Re: Ext4 vs XFS Filesystem - Survey of Popularity

dorio — Fri, 11 Dec 2015 01:42:43 GMT

@Neeraj Sabharwal

I had to ask to our customer, but i think there'll be no probelm

Re: Ext4 vs XFS Filesystem - Survey of Popularity

dennis_muller — Wed, 16 Dec 2015 23:09:09 GMT

Has anyone tried BTRFS?

We are testing with that now and have not seen any issues so far.

Re: Ext4 vs XFS Filesystem - Survey of Popularity

clove — Sat, 05 Mar 2016 03:29:33 GMT

Also same goes with LVM. I am thinking that LVM (without snapshots), and ext4 or xfs is good ...

Re: Ext4 vs XFS Filesystem - Survey of Popularity

gholmes — Fri, 31 Mar 2017 00:54:57 GMT

What are the recommended mount options for xfs?