Support Questions
Find answers, ask questions, and share your expertise

what is the maximum storage limit of hadoop

Highlighted

what is the maximum storage limit of hadoop

New Contributor
 
1 REPLY 1
Highlighted

Re: what is the maximum storage limit of hadoop

Mentor

@Raviteja Bhagam

The maximum storage in Hadoop also depends on the Namenode memory, as you are aware the name node MUST be able to store the metadata of all files in the cluster if you want to successfully read the heap size depends on many factors, such as the number of files, the number of blocks, and the load on the system.

See this HWX document Recommended NameNode Heap Size Settings ranging from 1million to 200 million files.

So coming back to your question cluster I know of a research centre with a 100 petabytes cluster in such a configuration the namenode memory should match the number of files. With such cheap prices for HDD's, you have flexibility.

HTH