About karthik_chandra

karthik_chandra · ‎06-26-2018

Thank you very much @Vinicius Higa Murakami

karthik_chandra · ‎06-21-2018

Hi @Vinicius Higa Murakami, Can you please let me know if I can delete the subdirNNN files under finalized folder? And how to permanently stop files being stored in finalized folder?

karthik_chandra · ‎06-20-2018

Hi @Vinicius Higa Murakami, Please find the attachment again.

karthik_chandra · ‎06-19-2018

Hi @Vinicius Higa Murakami, Please find below the value of dfs.datanode.du.reserved. <property> <name>dfs.datanode.du.reserved</name> <value>48204034048</value> </property> And regarding your question on subdir. Yes, i have a folder called subdir1 which is filled with this enormous blocks data. It is available in below path. /hadoop/hadoop/hdfs/data/current/BP-1468174578-IP_Address-1522297380740/current/finalized Mountpoints: lsblk

karthik_chandra · ‎06-14-2018

Hi @Vinicius Higa Murakami, Please also find the Disk space utilization in my system.

karthik_chandra · ‎06-14-2018

Hi @Vinicius Higa Murakami, Thanks again. Please find few of the screenshots below from my system and let me know if the way i am analyzing is right. DFS Storage: Storage description in Ambari: Non-DFS Storage: Here is where i have the blk_12345 files.

karthik_chandra · ‎06-13-2018

Hi @Vinicius Higa Murakami Thank you very much for the reply and the command to see where the block is being used. But what i don't understand is the file is already present in DFS and occupying specific amount of space. Why for the same file there is block on non-DFS also? Is it a kind of backup?

karthik_chandra · ‎06-12-2018

I am working in a 16 node cluster and recently i received an issue with the Non-DFS storage where the /hadoop/ mount which I am using is being consumed for non-DFS data and when I observed i found lot of blk_12345 etc. named .meta and original files. where each file is of 128 MB size and .meta file of size 1.1 MB (Totally all the files in total are consuming 1.7 TB of total cluster storage). Please let me know if I can remove these files and what is the impact if I remove them. And what is the reason they are created?

Online	Offline
Last Visited	‎06-26-2018 05:38 AM

Date Registered	‎06-12-2018 10:25 AM
Last Visited	‎06-26-2018 05:38 AM
Total Messages Posted	8

Re: Non-DFS storage occupied in Hadoop mount in Li...

Re: Non-DFS storage occupied in Hadoop mount in Li...

Re: Non-DFS storage occupied in Hadoop mount in Li...

Re: Non-DFS storage occupied in Hadoop mount in Li...

Re: Non-DFS storage occupied in Hadoop mount in Li...

Re: Non-DFS storage occupied in Hadoop mount in Li...

Re: Non-DFS storage occupied in Hadoop mount in Li...

Non-DFS storage occupied in Hadoop mount in Linux ...