Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Deletion of records from hive table based on some condition

Deletion of records from hive table based on some condition

New Contributor

Hi,

We have a requirement to delete records from hive table based on some condition.

and we need to make sure that in future if the data will come with same ID it should not be inserted.

For deletion one approach is to load the required data in another table delete the original one and rename the new one.

But how will we ensure that in future if the data will come with same ID it should not be inserted.

Please suggest some new and optimized approach for deletions if any.

Thanks.

3 REPLIES 3

Re: Deletion of records from hive table based on some condition

Contributor

Better to maintain all the deleted id's in one staging table and while loading the data into hive tables check whether id is already existed in staging table either by using Join or Merge clause

Re: Deletion of records from hive table based on some condition

Expert Contributor

if you are using 2.6 version or later, you can turn on ACID and execute delete commands. you can audit your delete via your app if it is needed. Otherwise, delete, merge and update can run on Hive directly with ACID.

Re: Deletion of records from hive table based on some condition

Contributor
Don't have an account?
Coming from Hortonworks? Activate your account here