Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Data Encryption and Decryption across the source and Target folders

Solved Go to solution
Highlighted

Data Encryption and Decryption across the source and Target folders

New Contributor

Hi I am new to Hadoop Platform and i have been asked to work on the data encryption and decryption.

Let me give you the background so that you can suggest me how to achieve this task. - I am sqooping the data from Teradata to my HDFS stg/src-a folder and creating external table on stg/src-a and pushing this data to my another layer tgt/input - this one is managed table, once data populated from stg/src-a to tgt/input, will cleanup stg/src-a folder.

whenever i bring data to stg/src-a, i need to left outer join with tgt/input data to load only newer record and wherever there is a match update to be triggered based on certain conditions. - question is, when i encrypt tgt/input folder it is not allowing to read data to join with stg/src-a external table.

Question 2: if the above one is successful, i need to expose the data to end-user by splitting total table and for some of the user i need to expose portion of the table by creating views or so. - If anyone of you worked on similar activites, please show some light on that. Thanks

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Data Encryption and Decryption across the source and Target folders

Expert Contributor

Are you using Hive? The only support engine for Hive to run with HDFS TDE is MR. Hive on Tez doesn't work yet today but will be in the near future. If you are using Hive on MR, can you post here the exception? What is your ranger policy defined for that encrypted folder?

5 REPLIES 5

Re: Data Encryption and Decryption across the source and Target folders

Master Collaborator

Are you using Transparent Data Encryption on HDFS?

Re: Data Encryption and Decryption across the source and Target folders

New Contributor

Yes Deepesh.

Re: Data Encryption and Decryption across the source and Target folders

Expert Contributor

Are you using Hive? The only support engine for Hive to run with HDFS TDE is MR. Hive on Tez doesn't work yet today but will be in the near future. If you are using Hive on MR, can you post here the exception? What is your ranger policy defined for that encrypted folder?

Re: Data Encryption and Decryption across the source and Target folders

New Contributor

Yes I am using Hive not Hive on Tez - I am not sure about the policy what has been defined.

Re: Data Encryption and Decryption across the source and Target folders

Expert Contributor

Are you still having issues babu?

Don't have an account?
Coming from Hortonworks? Activate your account here