question LUKS on HDFS in Archives of Support Questions (Read Only)

LUKS on HDFS

mike_garris — Tue, 08 Nov 2016 23:57:56 GMT

We had an issue come up where we can't use Ranger for encryption on HDFS and the question is can we use LUKS encryption instead?

Re: LUKS on HDFS

bhoomireddy_vij — Wed, 09 Nov 2016 00:22:50 GMT

@Mike Garris

Hi,

LUKS is a disk level encryption and hence is independent of the encryption supported by HDFS. Please see the link below to have an overview of the various levels of encryptions and where TDE sits.

https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html

Hope that answers your query.

Re: LUKS on HDFS

mike_garris — Wed, 09 Nov 2016 01:11:49 GMT

But this still relies on a backing KMS correct? Since we can't use Ranger, is there another method?

Re: LUKS on HDFS

mthiele1 — Wed, 09 Nov 2016 03:16:06 GMT

@Mike Garris Yes you can use LUKS as disk level encryption. This will encrypted the data blocks at the Linux level. This will not encrypted the data at the HDFS filesystem level. Many people have easily and successfully deployed HDFS with LUKS encrypted disk. The preference would to install and configure Linux and LUKS at the same time and then just install HDFS after as you would with a normal HDP install.

Re: LUKS on HDFS

mike_garris — Wed, 09 Nov 2016 03:24:18 GMT

Excellent.