Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

hive staging files

avatar
Expert Contributor

I have a spark job running using hivecontext which runs a query and writes it to orc table, i see a lot of hive staging files within the same directory of the hdfs location of the table like this : /.hive-staging_hive_2017-04-26_13-33-45_342_4121326216613322007-1

how to avoid this and best way to delete them?

2 REPLIES 2

avatar
@PJ

Try running set hive.exec.stagingdir=<new location> before running your query.

avatar
New Contributor

Is there a retention we can set for these staging directories in ambari? Seems like they are not cleaning up automatically