HDFS Encryption Data at Rest - in Non-Kerberized Environment

[#1]

Hi All,

I'm trying to set-up HDFS Encryption at Rest - in HDP 2.4 using Ranger KMS.

My cluster is Non-Kerberized, do i need to Kerberize the cluster before i can set-up HDFS Encryption ?

is that mandatory, or i can setup HDFS encryption in Non-Kerberized cluster also ?

Pls. note - the Docs mention kerberos setting (but not that Kerberos is mandatory)

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.3/bk_Security_Guide/content/ch06s01s01s01.htm...

Pls let me know.

[#2 · Accepted solution]

Without Kerberos, you don't have any authentication, hence no real security. Even if you encrypt the data, there's nothing to stop anyone talking to the cluster claiming to be the administrative user —so able to do lots of damage to the system.

Same for yarn: everything is executed in the cluster as the same user, so code by user Alice, running on the same host as user Bob, can use OS-level permissions and debuggers to get at all the secret's Bob's code has (including decryption keys)

I would recommend embracing Kerberos as the first step to having a secure cluster

[#3]

I agree with @stevel comment, I'd just add that encrypting the data at rest without Kerberos could only be useful in case disks are stolen. But if this is what you are trying to achieve it might be easier to rely on OS/disks native solutions.

[#4]

@stevel, @Pierre Villard - agreed.. i'll be using kerberos as first step, but still wanted to confirm if this was mandatory for hdfs encryption at rest.