Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Non-DFS storage occupied in Hadoop mount in Linux server

avatar

I am working in a 16 node cluster and recently i received an issue with the Non-DFS storage where the /hadoop/ mount which I am using is being consumed for non-DFS data and when I observed i found lot of blk_12345 etc. named .meta and original files. where each file is of 128 MB size and .meta file of size 1.1 MB (Totally all the files in total are consuming 1.7 TB of total cluster storage). Please let me know if I can remove these files and what is the impact if I remove them.

And what is the reason they are created?

1 ACCEPTED SOLUTION

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
14 REPLIES 14

avatar

Hey @Karthik Chandrashekhar!
Unfortunately, i'm not able to see the content of your attach, could you upload it again, please?

And regarding, your subdir, could you share the output from the following command:

du --max-depth=1 -h /hadoop/

Hope this helps!

avatar

Hi @Vinicius Higa Murakami,

Please find the attachment again.

77768-lsblk.png

avatar

Hi @Vinicius Higa Murakami,

Can you please let me know if I can delete the subdirNNN files under finalized folder?

And how to permanently stop files being stored in finalized folder?

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar

Thank you very much @Vinicius Higa Murakami