- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
hive staging files
- Labels:
-
Apache Hive
-
Apache Spark
Created ‎02-06-2018 03:55 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have a spark job running using hivecontext which runs a query and writes it to orc table, i see a lot of hive staging files within the same directory of the hdfs location of the table like this : /.hive-staging_hive_2017-04-26_13-33-45_342_4121326216613322007-1
how to avoid this and best way to delete them?
Created ‎02-07-2018 12:22 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Try running set hive.exec.stagingdir=<new location> before running your query.
Created ‎08-22-2018 10:03 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Is there a retention we can set for these staging directories in ambari? Seems like they are not cleaning up automatically
