Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HDF for teeing encrypted data?

avatar

Hi,

How does HDF handle teeing of encrypted data from PORD to DR?

Anything special hat needs to be done in terms of key management/decrypt process?

Thanks

1 ACCEPTED SOLUTION

avatar

@rbiswas

If you just use GetHDFS processor it should decrypt the data before moving it to DR (assuming it has the necessary read permissions), and once there, you can write it to an encryption zone. The data in-flight will be decrypted though.

Alternatively you can copy the data in it's encrypted form. You'd need to share the keys between the clusters and use HDF or DistCP to copy the files from the "/.reserved/..." folder rather than the regular folder. Take a look at the article below for clarification:

https://community.hortonworks.com/articles/51909/how-to-copy-encrypted-data-between-two-hdp-cluster....

View solution in original post

3 REPLIES 3

avatar

@rbiswas

If you just use GetHDFS processor it should decrypt the data before moving it to DR (assuming it has the necessary read permissions), and once there, you can write it to an encryption zone. The data in-flight will be decrypted though.

Alternatively you can copy the data in it's encrypted form. You'd need to share the keys between the clusters and use HDF or DistCP to copy the files from the "/.reserved/..." folder rather than the regular folder. Take a look at the article below for clarification:

https://community.hortonworks.com/articles/51909/how-to-copy-encrypted-data-between-two-hdp-cluster....

avatar
@Eyad Garelnabi

Thank you.

One question did you tested the GetHDFS processor for fetching from an encrypted zone?

avatar

I haven't tested it, but I don't believe it should be an issue since the decryption should happen transparently by the platform, before the data is passed to the processor.