Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to calculate hdp memory?

How to calculate hdp memory?

Expert Contributor

I use hdp and use the script from :

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_command-line-installation/content/determ...

But my server user raid5 ,so how to set the number of the disk?

4 REPLIES 4

Re: How to calculate hdp memory?

@darkz yu

Option 1: Search for dfs.datanode.data.dir in HDFS service in Ambari.

The values are separated by ,

Enter the number of directories specified over there.

Option 2:

You can also go to your terminal and issue:

sudo fdisk -l

and see the discs. Option 1 is easier though.

Thanks

Re: How to calculate hdp memory?

Expert Contributor

@rbiswas

I use the script to calc how to set the mem setting:

hdp-configuration-utils.py

from github https://github.com/hortonworks/hdp-configuration-utils

so a param is -d disk number.

My server had raided disks,so I don't how to determine how to set the disk number.

Set to 1 means the number of raid number,set 12 ,means the hard disk number.

Re: How to calculate hdp memory?

Super Guru
@darkz yu

Is this some sandbox cluster that you are setting up to play with Hadoop for learning purpose? If not, you should not be using RAID 5 for Hadoop disks. That's okay for OS and even okay for namenode if there is no HA/journal nodes. But you don't use RAID 5 for Hadoop data disks.

Re: How to calculate hdp memory?

Expert Contributor

I know what you mean,but my sa only support the server with RAID5 disks, so ...