Sharing Encryption Keys between clusters (replicating encrypted data)

[#1]

When using Ranger KMS and TDE is it possible to share encryption keys across 2 clusters? The scenario is that we have a Prod and DR cluster. When doing the data replication we'd like to avoid un-encrypting it on Prod, moving it over the wire, and then re-encrypting it when we write to DR. Is this possible?

[#2]

@Eyad Garelnabi This might be useful info, pls check - https://issues.apache.org/jira/browse/RANGER-749

[#3]

Thanks @Sagar Shimpi. I've seen this, but looking it the code it only seems like it's copying the master keys (EK). My understanding is that to un-encrypt a file you would need both, the master key (EK) stored in the DB as well as the file level encryption key (EDEK) which is store in the Name Node. Am I missing something or misunderstanding?

[#4]

Yes. It's possible. Update the same key on both KMS (prod and DR). I am using falcon to copy the data from prod to DR with KMS encryption.

[#5 · Accepted solution]

*Removed my previous response and adding the link to the article below:

https://community.hortonworks.com/articles/51909/how-to-copy-encrypted-data-between-two-hdp-cluster....

[#6]

In the second scenario, is it possible to copy the raw encrypted files from the first to the second cluster ?

[#7]

You would copy the file from "/.reserved/raw/test1/file1.txt" to "/.reserved/raw/test2/file1.txt" while preserving the extended attributes (where the EZEK is saved) using the -px flag.

https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html#Run...

https://issues.apache.org/jira/browse/MAPREDUCE-6007

[#8]

Is the link still working? I receive "Access Denied"...