question Re: Sharing Encryption Keys between clusters (replicating encrypted data) in Archives of Support Questions (Read Only)

Sharing Encryption Keys between clusters (replicating encrypted data)

egarelnabi — Tue, 16 Aug 2016 21:06:39 GMT

When using Ranger KMS and TDE is it possible to share encryption keys across 2 clusters? The scenario is that we have a Prod and DR cluster. When doing the data replication we'd like to avoid un-encrypting it on Prod, moving it over the wire, and then re-encrypting it when we write to DR. Is this possible?

Re: Sharing Encryption Keys between clusters (replicating encrypted data)

sshimpi — Tue, 16 Aug 2016 21:16:50 GMT

@Eyad Garelnabi This might be useful info, pls check - https://issues.apache.org/jira/browse/RANGER-749

Re: Sharing Encryption Keys between clusters (replicating encrypted data)

egarelnabi — Tue, 16 Aug 2016 21:24:14 GMT

Thanks @Sagar Shimpi. I've seen this, but looking it the code it only seems like it's copying the master keys (EK). My understanding is that to un-encrypt a file you would need both, the master key (EK) stored in the DB as well as the file level encryption key (EDEK) which is store in the Name Node. Am I missing something or misunderstanding?

Re: Sharing Encryption Keys between clusters (replicating encrypted data)

rbaskaran — Tue, 16 Aug 2016 21:49:07 GMT

Yes. It's possible. Update the same key on both KMS (prod and DR). I am using falcon to copy the data from prod to DR with KMS encryption.

Re: Sharing Encryption Keys between clusters (replicating encrypted data)

egarelnabi — Wed, 17 Aug 2016 00:14:12 GMT

*Removed my previous response and adding the link to the article below:

https://community.hortonworks.com/articles/51909/how-to-copy-encrypted-data-between-two-hdp-cluster.html

Re: Sharing Encryption Keys between clusters (replicating encrypted data)

jpp — Wed, 17 Aug 2016 01:56:20 GMT

In the second scenario, is it possible to copy the raw encrypted files from the first to the second cluster ?

Re: Sharing Encryption Keys between clusters (replicating encrypted data)

egarelnabi — Wed, 17 Aug 2016 02:25:02 GMT

You would copy the file from "/.reserved/raw/test1/file1.txt" to "/.reserved/raw/test2/file1.txt" while preserving the extended attributes (where the EZEK is saved) using the -px flag.

https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html#Running_as_the_superuser

https://issues.apache.org/jira/browse/MAPREDUCE-6007

Re: Sharing Encryption Keys between clusters (replicating encrypted data)

alpanchino — Wed, 10 Apr 2019 23:03:49 GMT

Is the link still working? I receive "Access Denied"...