- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Hive skip trash while doing insert overwrite on an external table
- Labels:
-
Apache Hive
-
HDFS
Created on ‎05-11-2017 12:40 AM - edited ‎09-16-2022 04:35 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am trying to skip the trash while doing insert overwrite on an external table from hive.
Tried using :
SET hive.warehouse.data.skiptrash=true;
ALTER TABLE TABLE_NAME DROP IF EXISTS partition(partition_value) PURGE;
Is there any alternative command from HIVE other than a hard delete on HDFS?
Created on ‎05-11-2017 08:00 PM - edited ‎05-11-2017 08:05 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
the other way around is to use em in Table properties , i belive it holds true for managed table .
not sure for external table , please try it let me know if you can .
TBLPROPERTIES ("auto.purge"="true") or ("auto.purge"="false") in release 1.2.0
Created ‎05-11-2017 11:37 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, this does not work even for managed tables, there is some bug logged.
Created ‎05-12-2017 05:01 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
i belive you are pointing this
HIVE-9118
its been fixed .
hive 1.2.0
Created ‎04-11-2018 02:58 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Am I right in reading the 5.13.3 and 5.14.2 tarball components? They both use Hive 1.1.0 and not Hive 1.2.0
I have a very similar problem - I want to switch off hive using the trash for certain tables
Created ‎06-27-2020 12:40 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Is ther any solution for this issue. I am facing same issue. Table is holding very huge data and while doing insert overwrite , files are getting placed in my user directory, /user/anjali/.Trash, causing hive action in oozie to fail after 1.5 hr long run. Please help. The table is external and ev even I changed it to internal table, auto purge = true is not working.
