Cloudera Community
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
rajasuba_s
user rank
Explorer
My Accepted Solutions
Show All

Re: There is insufficient memory for the Java Runt...

by Explorer in Support Questions
‎09-12-2018 10:24 AM
‎09-12-2018 10:24 AM
@Joshua Adeleke: I'm asking this out of curiosity, how does reducing heap space would provide enough memory to create a new thread? ... View more

Re: HDFS is almost full 90% but data node disks ar...

by Explorer in Support Questions
‎09-05-2018 10:01 AM
‎09-05-2018 10:01 AM
@Karthik Palanisamy : I'm sorry to be wrong. Consider two blocks (64 MB, 64MB) and only I've used only (2MB, 3MB) in each block. Means can I reuse the remaining (123MB) to another one block of 64MB? ... View more

Re: HDFS is almost full 90% but data node disks ar...

by Explorer in Support Questions
‎09-04-2018 10:54 AM
‎09-04-2018 10:54 AM
Yes possibly @Michael Bronson. You could also check "Trash" files size as @Geoffrey Shelton Okot suggests. In general it is advised not to store too many small files in HDFS. In general, HDFS is good for storing large files. ... View more

Re: HDFS is almost full 90% but data node disks ar...

by Explorer in Support Questions
‎09-04-2018 10:16 AM
‎09-04-2018 10:16 AM
HDFS originally splits and stores the data in blocks. Each block is 64MB or 128MB by default based on your HDFS version. Consider a file which is of size (2MB) is stored in a block. The remaining 62MB (considering the default size to be 64MB) is not used by HDFS. Which means here the HDFS used space is 64MB but the actual hard disk used space is 2MB. Hope this is relevant to what you are asking. ... View more

Re: How hdfs generates datanode path?

by Explorer in Support Questions
‎08-21-2018 10:26 AM
‎08-21-2018 10:26 AM
Why does this again creating "current/finalized/subdir0/subdir0" ? ... View more

Re: How hdfs generates datanode path?

by Explorer in Support Questions
‎08-21-2018 10:24 AM
‎08-21-2018 10:24 AM
Combining @hdpadmin 's answer and @Jonathan Sneep's answer gives me the best answer to my question 🙂 Thank you both 🙂 ... View more

Re: How hdfs generates datanode path?

by Explorer in Support Questions
‎08-21-2018 10:22 AM
‎08-21-2018 10:22 AM
This is exactly what I was looking for ! Thanks a lot @Jonathan Sneep ... View more

Re: How hdfs generates datanode path?

by Explorer in Support Questions
‎08-21-2018 09:28 AM
‎08-21-2018 09:28 AM
Thanks @hdpadmin. You were right I have configured "/data/hadoop-data/dn" in hadoop-site.xml. But why does hdfs create multiple subfolders within the current folder like "BP-1308070615-172.22.131.23-1533215887051/current/finalized/subdir0/subdir0"? What does the sub directory name mean? ... View more

How hdfs generates datanode path?

by Explorer in Support Questions
‎08-20-2018 12:59 PM
‎08-20-2018 12:59 PM
Reference : https://stackoverflow.com/questions/51930336/how-datanode-path-is-created-in-hadoop How do hdfs generates datanode path? Say for example I'm asking hdfs to write a file in hdfs location hdfs://192.168.143.150:9000/filestore/TAO.mp4 it is getting written in the path "/data/hadoop-data/dn/current/BP-1308070615-172.22.131.23-1533215887051/current/finalized/subdir0/subdir0/blk_1073741869". How this path gets generated? ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎09-20-2018 08:46 AM
Community Statistics
Member Since ‎08-10-2018 07:39 AM
Last Visited ‎09-20-2018 08:46 AM
Posts 12