Support Questions

Find answers, ask questions, and share your expertise

Cleaning Hive temp directory -Permissions problems in CDP Public Cloud

avatar

Hello,

due to a problem with a script, we have almost saturated the HDFS available space.

I suppose this is caused by temporary Hive files that have not been cleaned up due to the abnormal termination of the script.

I would like to check the /tmp/hive folder on HDFS but my user, that has administrative privileges, cannot access the /tmp/hive folder.

Is there a way to check and clean such folder?

 

Any help would be appreciated.

 

KRs,

Andrea

1 ACCEPTED SOLUTION

avatar
New Contributor

Hi Andrea,

use hdfs keytab to clear them 

NAME=hdfs;
KEYTAB=$(find /run/cloudera-scm-agent/process -name ${NAME}.keytab -path "*${NAME}-*" | sort | tail -n 1);
PRINCIPAL=$(klist -kt "$KEYTAB" | awk '{ print $4 }' | grep "^${NAME}" | head -n 1);
kinit -kt "${KEYTAB}" "${PRINCIPAL}"

Thanks,

Vinay

View solution in original post

4 REPLIES 4

avatar
Expert Contributor

Hi Andrea,

 

Is this a kerberized environment?


Best,

-JMP

avatar

Hi JM, yes it's a kerberized environment.

 

KRs,

Andrea

avatar
Expert Contributor
Andrea,

On the node running HS2, locate the most recent hive.keytab using
```
ls -ltr
/var/run/cloudera-scm-agent/process/*hive_on_tez-HIVESERVER2/hive.keytab |
tail -1
```
Then kinit as `hive/@REALM`
After that, you should be able to scan directories owned by the Hive user
```
hdfs dfs -ls /tmp/hive
```
Hope this helps
-JMP

avatar
New Contributor

Hi Andrea,

use hdfs keytab to clear them 

NAME=hdfs;
KEYTAB=$(find /run/cloudera-scm-agent/process -name ${NAME}.keytab -path "*${NAME}-*" | sort | tail -n 1);
PRINCIPAL=$(klist -kt "$KEYTAB" | awk '{ print $4 }' | grep "^${NAME}" | head -n 1);
kinit -kt "${KEYTAB}" "${PRINCIPAL}"

Thanks,

Vinay