As with many topics, "it depends". For slave/worker/data hosts which only have distributed services you can likely disable swap. With distributed services it's preferred to let the process/host be killed rather than swap. The killing of that process or host shouldn't affect cluster availability. Said another way: you want to "fail fast" not to "slowly degrade". Just 1 bad process/host can greatly degrade performance of the whole cluster. For example, in a 350 host cluster removal of 2 bad nodes improved throughput by ~2x: http://www.slideshare.net/t3rmin4t0r/tez8-ui-walkthrough/23 http://pages.cs.wisc.edu/~thanhdo/pdf/talk-socc-limplock.pdf For masters, swap is also often disabled though it's not a set rule from Hortonworks and I assume there will be some discussion/disagreement. Masters can be treated somewhat like you'd treat masters in other, non-Hadoop, environments. The fear with disabling swap on masters is that an OOM (out of memory) event could affect cluster availability. But that will still happen even with swap configured, it just will take slightly longer. Good administrator/operator practices would be to monitor RAM availability, then fix any issues before running out of memory. Thus maintaining availability without affecting performance. No swap is needed then. Scenarios where you might want swap: playing/testing functionality, not performance, on hosts with very little RAM so will likely need to swap. if you have the need to use more memory, or expect to need more, than the amount of RAM which has been purchased. And can accept severe degradation in failure. In this case you would need a lot of swap configured. Your better off buying the right amount of memory. Extra thoughts: if you want to disable swap, but your organization require their to be a swap partition, set swappiness=0 if you choose to have swap, set swappiness=1 to avoid swapping until all physical memory has been used. most Cloud/Virtualization providers disable swap by default. Don't change that. some advise to avoid swap on SSDs due to reducing their lifespan
... View more
