Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Having issue with /tmp directory removal

Highlighted

Having issue with /tmp directory removal

Rising Star

Hello

I need to remove /tmp/hive/hive directory content which has too many folders which i guess most of them are empty.

bash-4.2$ hadoop fs -count /tmp/hive/hive
2097194 18 2581710 /tmp/hive/hive

I tried using a script which referred before in below question

https://community.hortonworks.com/questions/21684/today-tfh-refused-to-start-with-message-of-orgapac...

Script is located in Github:

https://github.com/nmilford/clean-hadoop-tmp

Script is written in Ruby. When i run this script I receive an exception. How can I delete them? Do you have any suggestions?

Exception with Ruby script is:

-bash-4.2$ ./clean-hadoop-tmp
Dropping lock file. (/var/tmp/clean-hadoop-tmp.lock)
Scanning for directories in HDFS' /tmp older than 1800 seconds.
./clean-hadoop-tmp:38:in `block in <main>': undefined method `each' for nil:NilClass (NoMethodError)
from ./clean-hadoop-tmp:35:in `each'
from ./clean-hadoop-tmp:35:in `<main>'


Should I modify this script and how? Is there any other ways?

4 REPLIES 4

Re: Having issue with /tmp directory removal

Rising Star

Initial solution i tried was a script in below answer

https://community.hortonworks.com/answers/92110/view.html

I modified this script to work in /tmp/hive/hive path but it received OOM error.

Re: Having issue with /tmp directory removal

Contributor

@Sedat Kestepe

There is a empty directory inside /tmp/

Just modify the code like this and it will work.

target_dirs.each do |tdir|
    target_dir = tdir.split(" ")[7]
    sub_dirs = []
    dir_list = `hadoop fs -ls #{target_dir}`.split("\n")[2..-1]
    sub_dirs = dir_list if dir_list
    sub_dirs.each do |sdir|

Re: Having issue with /tmp directory removal

Rising Star

Good idea. Felt bad and lazy for not inspecting the code before.

Got again OOM however.

Thank you anyways. I will try to spare some time to take a look at the code. I will update the question if I can progress.

Re: Having issue with /tmp directory removal

Rising Star

What I did was editing the code giving '/tmp/hive/hive' as the root directory. I should have given '/tmp' at the first place.

Don't have an account?
Coming from Hortonworks? Activate your account here