Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDF for teeing encrypted data?

Solved Go to solution
Highlighted

HDF for teeing encrypted data?

Hi,

How does HDF handle teeing of encrypted data from PORD to DR?

Anything special hat needs to be done in terms of key management/decrypt process?

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: HDF for teeing encrypted data?

@rbiswas

If you just use GetHDFS processor it should decrypt the data before moving it to DR (assuming it has the necessary read permissions), and once there, you can write it to an encryption zone. The data in-flight will be decrypted though.

Alternatively you can copy the data in it's encrypted form. You'd need to share the keys between the clusters and use HDF or DistCP to copy the files from the "/.reserved/..." folder rather than the regular folder. Take a look at the article below for clarification:

https://community.hortonworks.com/articles/51909/how-to-copy-encrypted-data-between-two-hdp-cluster....

View solution in original post

3 REPLIES 3
Highlighted

Re: HDF for teeing encrypted data?

@rbiswas

If you just use GetHDFS processor it should decrypt the data before moving it to DR (assuming it has the necessary read permissions), and once there, you can write it to an encryption zone. The data in-flight will be decrypted though.

Alternatively you can copy the data in it's encrypted form. You'd need to share the keys between the clusters and use HDF or DistCP to copy the files from the "/.reserved/..." folder rather than the regular folder. Take a look at the article below for clarification:

https://community.hortonworks.com/articles/51909/how-to-copy-encrypted-data-between-two-hdp-cluster....

View solution in original post

Highlighted

Re: HDF for teeing encrypted data?

@Eyad Garelnabi

Thank you.

One question did you tested the GetHDFS processor for fetching from an encrypted zone?

Highlighted

Re: HDF for teeing encrypted data?

I haven't tested it, but I don't believe it should be an issue since the decryption should happen transparently by the platform, before the data is passed to the processor.

Don't have an account?
Coming from Hortonworks? Activate your account here