is it safe to remove the /tmp/hive/hive folder?

Question

is it safe to remove the /tmp/hive/hive folder? ( from hdfs )ashdfs dfs -rm -r /tmp/hive/hivethe reason for that because under /tmp/hive/hive we have thousand  of files and we cant delete therm hdfs dfs -ls /tmp/hive/Found 7 itemsdrwx------   - admin     hdfs          0 2019-03-05 12:00 /tmp/hive/admindrwx------   - drt       hdfs          0 2019-06-16 14:02 /tmp/hive/drtdrwx------   - ambari-qa hdfs          0 2019-06-16 15:11 /tmp/hive/ambari-qadrwx------   - anonymous hdfs          0 2019-06-16 08:57 /tmp/hive/anonymousdrwx------   - hdfs      hdfs          0 2019-06-13 08:42 /tmp/hive/hdfsdrwx------   - hive      hdfs          0 2019-06-13 10:58 /tmp/hive/hivedrwx------   - root      hdfs          0 2018-07-17 23:37 /tmp/hive/rootYou have mail in /var/spool/mail/root

jsensharma · Accepted Answer

@Michael BronsonAs per the apache hive docs there seems to be some parameters and tools available to deal with such issue. Although i have not personally tested those tools. But looks like they were introduced to deal with similar issue long back as part of https://issues.apache.org/jira/browse/HIVE-13429For example i see that the Hive Config "hive.exec.scratchdir" points to the "/tmp/hive" dir.Can you please check and let us know what is the value set for the following parameter "hive.scratchdir.lock". (if not set then default value will be "false"? Additionally you might want to refer about "hive.server2.clear.dangling.scratchdir" and "hive.start.cleanup.scratchdir" parameters of Hive Server config.Please refer to [1] the following link to know more about those parameters.There is a tool "cleardanglingscratchdir" mentioned as part of the link [2] may be you would like to read more about it.# hive --service cleardanglingscratchdir [-r] [-v] [-s scratchdir]    -r      dry-run mode, which produces a list on console    -v      verbose mode, which prints extra debugging information    -s      if you are using non-standard scratch directory.[1] https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.scratchdir.lock.[2] https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2#SettingUpHiveServer2-ClearDanglingScratchDirTool

mike_bronson7 · Answer

we try the following to remove files that older then 10 days , but because there are so many files then files not deleted at all

hdfs dfs -ls /tmp/hive/hive   |   tr -s " "    |    cut -d' ' -f6-8    |     grep "^[0-9]"    |    awk 'BEGIN{ MIN=14400; LAST=60*MIN; "date +%s" | getline NOW } { cmd="date -d'\''"$1" "$2"'\'' +%s"; cmd | getline WHEN; DIFF=NOW-WHEN; if(DIFF > LAST){ print "Deleting: "$3; system("hdfs dfs -rm -r "$3) }}'

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.