Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HIVE + how to avoid folders creation under /tmp or at least to minimized folder creation

HIVE + how to avoid folders creation under /tmp or at least to minimized folder creation

hi all

we found more then 14000 folders under /tmp with owned by HIVE

the folders are like this

{session_id}_resources

actually we can create cron Job that will responsible to delete each day the folders that are older then 1 hour

but I don't love this approach

can we avoid this folders creation by some setting on the ambari ?

or at least to minimize the folders creation ?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Reference:

Taken from Hive Documentation:

“Temporary Folders

Hive uses temporary folders both on the machine running the Hive client and the default HDFS instance. These folders are used to store per-query temporary/intermediate data sets and are normally cleaned up by the hive client when the query is finished. However, in cases of abnormal hive client termination, some data may be left behind. The configuration details are as follows:

On the HDFS cluster this is set to /tmp/hive-<username> by default and is controlled by the configuration variable hive.exec.scratchdir

On the client machine, this is hardcoded to /tmp/<username>

Note that when writing data to a table/partition, Hive will first write to a temporary location on the target table's filesystem (using hive.exec.scratchdir as the temporary location) and then move the data to the target table. This applies in all cases - whether tables are stored in HDFS (normal case) or in file systems like S3 or even NFS.”

Michael-Bronson
Don't have an account?
Coming from Hortonworks? Activate your account here