Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDFS disk allocation

Solved Go to solution
Highlighted

HDFS disk allocation

Master Collaborator

does the location of these two parameters tell where HDFS will be using its space from?

NameNode directories=/u01/hadoop/hdfs/namenode 
DataNode directories=/u01/hadoop1/hdfs/data

also if i want to add more space to the cluster by adding another disk , how do i add it to the cluster?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: HDFS disk allocation

@Sami Ahmad Datanode directories is where HDFS data is actually stored while Namenode Directory is used for storing Namenode Metadata i.e. file, block information etc. associated with HDFS data stored on Datanodes. To add additional space to HDFS with additional disks, you should add disk to Datanode Directories i.e. if you add u02 to server(s), add the same to Datanode Directories.

DataNode directories=/u01/hadoop1/hdfs/data,/u02/hadoop1/hdfs/data
3 REPLIES 3

Re: HDFS disk allocation

@Sami Ahmad Datanode directories is where HDFS data is actually stored while Namenode Directory is used for storing Namenode Metadata i.e. file, block information etc. associated with HDFS data stored on Datanodes. To add additional space to HDFS with additional disks, you should add disk to Datanode Directories i.e. if you add u02 to server(s), add the same to Datanode Directories.

DataNode directories=/u01/hadoop1/hdfs/data,/u02/hadoop1/hdfs/data

Re: HDFS disk allocation

Master Collaborator

thanks Pardeep.

so should we keep the HDFS disk seperate from the OS disk , i.e where the linux operating system is installed ?

Re: HDFS disk allocation

Yes that is recommended approach.

Don't have an account?
Coming from Hortonworks? Activate your account here