Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Deletion of records from hive table based on some condition

avatar
New Contributor

Hi,

We have a requirement to delete records from hive table based on some condition.

and we need to make sure that in future if the data will come with same ID it should not be inserted.

For deletion one approach is to load the required data in another table delete the original one and rename the new one.

But how will we ensure that in future if the data will come with same ID it should not be inserted.

Please suggest some new and optimized approach for deletions if any.

Thanks.

3 REPLIES 3

avatar
Rising Star

Better to maintain all the deleted id's in one staging table and while loading the data into hive tables check whether id is already existed in staging table either by using Join or Merge clause

avatar
Super Collaborator

if you are using 2.6 version or later, you can turn on ACID and execute delete commands. you can audit your delete via your app if it is needed. Otherwise, delete, merge and update can run on Hive directly with ACID.

avatar
Rising Star