LUKS on HDFS

[#1]

We had an issue come up where we can't use Ranger for encryption on HDFS and the question is can we use LUKS encryption instead?

[#2]

@Mike Garris

Hi,

LUKS is a disk level encryption and hence is independent of the encryption supported by HDFS. Please see the link below to have an overview of the various levels of encryptions and where TDE sits.

https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html

Hope that answers your query.

[#3]

But this still relies on a backing KMS correct? Since we can't use Ranger, is there another method?

[#4 · Accepted solution]

@Mike Garris Yes you can use LUKS as disk level encryption. This will encrypted the data blocks at the Linux level. This will not encrypted the data at the HDFS filesystem level. Many people have easily and successfully deployed HDFS with LUKS encrypted disk. The preference would to install and configure Linux and LUKS at the same time and then just install HDFS after as you would with a normal HDP install.

[#5]

Excellent.