Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Datanodes with different amount and size of physical disks/hdfs partitions.

Highlighted

Datanodes with different amount and size of physical disks/hdfs partitions.

New Contributor

Hello,

At the organization I work for we have a hadoop cluster. We recently purchased new datanodes to expand the cluster. The new datanodes have 12 disks (8TB) instead of 10 (6TB), as the older datanodes. 

After some research (mainly this community) I came to understand that its not a requirement to have the same number of drives in each node or equally sized disks. 

On each old datanode we have 8 hdfs partitions one for every disk.

Should we create 10 hdfs partitions on each new datanode or create 8 volumes using the 10 disks?

Should we utilize all the available storage?

And if we do is there something specific we should monitor?

 

1 REPLY 1
Highlighted

Re: Datanodes with different amount and size of physical disks/hdfs partitions.

New Contributor

Correction:

On each old datanode we have 10 hdfs partitions one for every disk.

Should we create 12 hdfs partitions on each new datanode or create 10 volumes using the 12 disks?

Don't have an account?
Coming from Hortonworks? Activate your account here