Member since
08-10-2018
12
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4394 | 08-21-2018 10:24 AM |
09-12-2018
10:24 AM
@Joshua Adeleke: I'm asking this out of curiosity, how does reducing heap space would provide enough memory to create a new thread?
... View more
09-05-2018
10:01 AM
@Karthik Palanisamy : I'm sorry to be wrong. Consider two blocks (64 MB, 64MB) and only I've used only (2MB, 3MB) in each block. Means can I reuse the remaining (123MB) to another one block of 64MB?
... View more
09-04-2018
10:54 AM
Yes possibly @Michael Bronson. You could also check "Trash" files size as @Geoffrey Shelton Okot suggests. In general it is advised not to store too many small files in HDFS. In general, HDFS is good for storing large files.
... View more
09-04-2018
10:16 AM
HDFS originally splits and stores the data in blocks. Each block is 64MB or 128MB by default based on your HDFS version. Consider a file which is of size (2MB) is stored in a block. The remaining 62MB (considering the default size to be 64MB) is not used by HDFS. Which means here the HDFS used space is 64MB but the actual hard disk used space is 2MB. Hope this is relevant to what you are asking.
... View more
08-21-2018
10:26 AM
Why does this again creating "current/finalized/subdir0/subdir0" ?
... View more
08-21-2018
10:24 AM
Combining @hdpadmin 's answer and @Jonathan Sneep's answer gives me the best answer to my question 🙂 Thank you both 🙂
... View more
08-21-2018
10:22 AM
This is exactly what I was looking for ! Thanks a lot @Jonathan Sneep
... View more
08-21-2018
09:28 AM
Thanks @hdpadmin. You were right I have configured "/data/hadoop-data/dn" in hadoop-site.xml. But why does hdfs create multiple subfolders within the current folder like "BP-1308070615-172.22.131.23-1533215887051/current/finalized/subdir0/subdir0"? What does the sub directory name mean?
... View more
08-20-2018
12:59 PM
Reference : https://stackoverflow.com/questions/51930336/how-datanode-path-is-created-in-hadoop How do hdfs generates datanode path? Say for example I'm asking hdfs to write a file in hdfs location hdfs://192.168.143.150:9000/filestore/TAO.mp4 it is getting written in the path "/data/hadoop-data/dn/current/BP-1308070615-172.22.131.23-1533215887051/current/finalized/subdir0/subdir0/blk_1073741869". How this path gets generated?
... View more
Labels:
- Labels:
-
Apache Hadoop