While the sort is running I noticed that many GB of disks are used by at /hadoop/yarn/local (that's: yarn.nodemanager.local-dirs). It means lots of writing and reading to a physical disk other than the files to sort.
In addition, I see that most of my 256GB of RAM is used by buff/cache. Is this the most efficient usage of the memory I have?
In order to use more RAM for sorting to speed up the processing, I tried to tune various values for: