The Google Cloud Storage Connector for Hadoop is configured at the cluster level without any knowledge of Kerberos. So the output you showed is what I would expect. But some thoughts: In secure environments, ideally a user can never even reach Hadoop without authentication against the Kerberos or Directory. With that assumed, you would never get the chance to run 'hadoop fs -ls ...' anyway. So lock down all access to the environment & network so only authorized users can even run the commands. It couldn't hurt to submit a feature request for a configuration option that disables 'gs' unless the user is authenticated to Hadoop. Personally I see this as a bug report, but technically it's a feature request. You would have to raise it with Google since the Connector is not currently a part of Apache Hadoop. Google maintains it separately. Why it's not a bug: Kerberos governs communications between services, not the executions of commands. Since GS doesn't do Kerberos, it works as intended since it already has it's authentication done separately. I've not done it, but you could check if individual users/applications can pass the GCS token. If possible then you would remove it from the cluster-wide configuration and the users would be required to do this themselves. It would still not be using Kerberos but would be another layer of security. s3a://, swift://, and wasb:// support this method. ... View more

As with many topics, "it depends". For slave/worker/data hosts which only have distributed services you can likely disable swap. With distributed services it's preferred to let the process/host be killed rather than swap. The killing of that process or host shouldn't affect cluster availability. Said another way: you want to "fail fast" not to "slowly degrade". Just 1 bad process/host can greatly degrade performance of the whole cluster. For example, in a 350 host cluster removal of 2 bad nodes improved throughput by ~2x: http://www.slideshare.net/t3rmin4t0r/tez8-ui-walkthrough/23 http://pages.cs.wisc.edu/~thanhdo/pdf/talk-socc-limplock.pdf For masters, swap is also often disabled though it's not a set rule from Hortonworks and I assume there will be some discussion/disagreement. Masters can be treated somewhat like you'd treat masters in other, non-Hadoop, environments. The fear with disabling swap on masters is that an OOM (out of memory) event could affect cluster availability. But that will still happen even with swap configured, it just will take slightly longer. Good administrator/operator practices would be to monitor RAM availability, then fix any issues before running out of memory. Thus maintaining availability without affecting performance. No swap is needed then. Scenarios where you might want swap: playing/testing functionality, not performance, on hosts with very little RAM so will likely need to swap. if you have the need to use more memory, or expect to need more, than the amount of RAM which has been purchased. And can accept severe degradation in failure. In this case you would need a lot of swap configured. Your better off buying the right amount of memory. Extra thoughts: if you want to disable swap, but your organization require their to be a swap partition, set swappiness=0 if you choose to have swap, set swappiness=1 to avoid swapping until all physical memory has been used. most Cloud/Virtualization providers disable swap by default. Don't change that. some advise to avoid swap on SSDs due to reducing their lifespan ... View more