Logical Disk Encryption - Data at Rest Encryption

[#1]

Logical Disk Encryption

The approach is to avoid using HDFS encryption and use Disk LUKS encryption for data at rest encryption requirement specially when using public cloud IAAS.

To build manually encrypted volumes or drives ,use following steps on a d2*8xLarge instance flavor.

lsblk
cryptsetup --verbose --verify-passphrase luksFormat /dev/xvdb
cryptsetup --verbose --verify-passphrase luksFormat /dev/xvdc
cryptsetup --verbose --verify-passphrase luksFormat /dev/xvdd

cryptsetup luksOpen /dev/xvdb vol1
cryptsetup luksOpen /dev/xvdc vol2
cryptsetup luksOpen /dev/xvdd vol3

dd if=/dev/urandom of=/root/keyfile1 bs=1024 count=4
chmod 0400 /root/keyfile1

cryptsetup luksAddKey /dev/xvdb /root/keyfile1
cryptsetup luksAddKey /dev/xvdc /root/keyfile1
cryptsetup luksAddKey /dev/xvdd /root/keyfile1

mkfs.ext4 /dev/mapper/vol1
mkfs.ext4 /dev/mapper/vol2
mkfs.ext4 /dev/mapper/vol3

echo "/dev/mapper/vol1 /data/vol1 ext4 defaults,nofail,nodev 0 2" >> /etc/fstab
echo "/dev/mapper/vol2 /data/vol2 ext4 defaults,nofail,nodev 0 2" >> /etc/fstab
echo "/dev/mapper/vol3 /data/vol3 defaults,nofail,nodev 0 2" >> /etc/fstab

echo "vol1 /dev/xvdb /root/keyfile1 luks" >>/etc/crypttab
echo "vol1 /dev/xvdc /root/keyfile1 luks" >>/etc/crypttab
echo "vol1 /dev/xvdd /root/keyfile1 luks" >>/etc/crypttab

mount -a

Automated Shell Script

#!/bin/bash

set -x

set: 

${PLATFORM_DISK_PREFIX:? required}: 

${START_LABEL:? required}

format_disks_encrypted()
{ 
  mkdir /hadoopfs 
  openssl rand -base64 32 > /root/encrypt 
  cat /root/encrypt  > /root/encrypt1 
  cat /root/encrypt1 > /root/encrypt2 
  yum -y install  cryptsetup-luks 

for (( i=1; i<=24;i++ )); do  
   LABEL=$(printf "\x$(printf %x $((START_LABEL+i)))")  
   DEVICE=/dev/${PLATFORM_DISK_PREFIX}${LABEL}  
  if [ -e $DEVICE ]; then  
     MOUNTPOINT=$(grep $DEVICE /etc/fstab | tr -s ' \t' ' ' | cut -d' ' -f 2)  
     if [ -n "$MOUNTPOINT" ]; then  
       umount "$MOUNTPOINT"  
       sed -i "\|^$DEVICE|d" /etc/fstab  
    fi  
  mkdir /hadoopfs/fs${i}  
  cryptsetup --verbose luksFormat $DEVICE -yrq --key-file=/root/encrypt 
  cryptsetup luksOpen $DEVICE vol${i} --key-file=/root/encrypt  
  cryptsetup luksAddKey $DEVICE /root/encrypt --key-file=/root/encrypt  
  mkfs.ext4  /dev/mapper/vol${i}  
  echo UUID=$(blkid -o value /dev/mapper/vol${i} | head -1) /hadoopfs/fs${i} ext4 inode_readahead_blks=128,data=writeback,noatime,nodiratime 0 2 >> /etc/fstab  
  echo "vol${i} `UUID=$(blkid -o value /dev/mapper/vol${i} | head -1)` /root/encrypt luks" >> /etc/crypttab
  mount /hadoopfs/fs${i}
  chmod 777 /hadoopfs/fs${i}
fi 

done
}

main() 
{  
format_disks_encrypted
}

Reference: https://wiki.archlinux.org/index.php/Dm-crypt/Device_encryption