Support Questions
Find answers, ask questions, and share your expertise

Having issue with /tmp directory removal

Rising Star


I need to remove /tmp/hive/hive directory content which has too many folders which i guess most of them are empty.

bash-4.2$ hadoop fs -count /tmp/hive/hive
2097194 18 2581710 /tmp/hive/hive

I tried using a script which referred before in below question

Script is located in Github:

Script is written in Ruby. When i run this script I receive an exception. How can I delete them? Do you have any suggestions?

Exception with Ruby script is:

-bash-4.2$ ./clean-hadoop-tmp
Dropping lock file. (/var/tmp/clean-hadoop-tmp.lock)
Scanning for directories in HDFS' /tmp older than 1800 seconds.
./clean-hadoop-tmp:38:in `block in <main>': undefined method `each' for nil:NilClass (NoMethodError)
from ./clean-hadoop-tmp:35:in `each'
from ./clean-hadoop-tmp:35:in `<main>'

Should I modify this script and how? Is there any other ways?


Rising Star

Initial solution i tried was a script in below answer

I modified this script to work in /tmp/hive/hive path but it received OOM error.


@Sedat Kestepe

There is a empty directory inside /tmp/

Just modify the code like this and it will work.

target_dirs.each do |tdir|
    target_dir = tdir.split(" ")[7]
    sub_dirs = []
    dir_list = `hadoop fs -ls #{target_dir}`.split("\n")[2..-1]
    sub_dirs = dir_list if dir_list
    sub_dirs.each do |sdir|

Rising Star

Good idea. Felt bad and lazy for not inspecting the code before.

Got again OOM however.

Thank you anyways. I will try to spare some time to take a look at the code. I will update the question if I can progress.

Rising Star

What I did was editing the code giving '/tmp/hive/hive' as the root directory. I should have given '/tmp' at the first place.

; ;