question Re: Cleaning Hive temp directory -Permissions problems in CDP Public Cloud in Support Questions

Cleaning Hive temp directory -Permissions problems in CDP Public Cloud

AndreaCavenago — Thu, 02 Jun 2022 16:41:12 GMT

Hello,

due to a problem with a script, we have almost saturated the HDFS available space.

I suppose this is caused by temporary Hive files that have not been cleaned up due to the abnormal termination of the script.

I would like to check the /tmp/hive folder on HDFS but my user, that has administrative privileges, cannot access the /tmp/hive folder.

Is there a way to check and clean such folder?

Any help would be appreciated.

KRs,

Andrea

Re: Cleaning Hive temp directory -Permissions problems in CDP Public Cloud

JoseManuel — Thu, 02 Jun 2022 19:00:14 GMT

Hi Andrea,

Is this a kerberized environment?

Best,

-JMP

Re: Cleaning Hive temp directory -Permissions problems in CDP Public Cloud

AndreaCavenago — Thu, 02 Jun 2022 21:51:32 GMT

Hi JM, yes it's a kerberized environment.

KRs,

Andrea

Re: Cleaning Hive temp directory -Permissions problems in CDP Public Cloud

JoseManuel — Mon, 06 Jun 2022 21:51:25 GMT

Andrea,

On the node running HS2, locate the most recent hive.keytab using
```
ls -ltr
/var/run/cloudera-scm-agent/process/*hive_on_tez-HIVESERVER2/hive.keytab |
tail -1
```
Then kinit as `hive/@REALM`
After that, you should be able to scan directories owned by the Hive user
```
hdfs dfs -ls /tmp/hive
```
Hope this helps
-JMP

Re: Cleaning Hive temp directory -Permissions problems in CDP Public Cloud

vinayreddyperam — Tue, 07 Jun 2022 17:45:09 GMT

Hi Andrea,

use hdfs keytab to clear them

NAME=hdfs;
KEYTAB=$(find /run/cloudera-scm-agent/process -name ${NAME}.keytab -path "*${NAME}-*" | sort | tail -n 1);
PRINCIPAL=$(klist -kt "$KEYTAB" | awk '{ print $4 }' | grep "^${NAME}" | head -n 1);
kinit -kt "${KEYTAB}" "${PRINCIPAL}"

Thanks,

Vinay