All OS settings for RedHat Cluster?

[#1]

Hi,

I'm planning to install the Ambari 2.1.1 HDP 2.4 on RedHat 7.2 Operating System. Does anyone have a complete checklist of OS settings need to be done before installation? I need something other than this link

Example:

vm.swappniess=0

vm.dirty_ratio=0

vm.dirty_background_ratio=0

Changing THP to never

Thanks in advance for providing any information.

[#2]

@SBandaru

I think this link is what you are looking for.

http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.0/bk_Installing_HDP_AMB/content/_prepare_the_e...

Also, I am pretty sure in the newer version of Redhat, you need to setup swappiness to 1 instead of 0. I would also disable transparent_hugepage

do a cat on the the following file on your OS and see if it's set to never or always. If it's at always, change it to never.

/sys/kernel/mm/redhat_transparent_hugepage/defrag

Use the following command to change this value to never

echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag

I am not hundred percent sure but I think this requires a restart.

[#3]

@mqureshi

Thanks for the quick response, I have already made all the changes which are in that link. Even I have made changes which I mentioned in the example. Now I need more OS level setting which improves cluster performance.

[#4]

sure. change swappiness to 1 not 0 which is what you have. Also disable transparent_huge_page as well as transparent_hugepage defrag.

When you are using Hadoop, your files are in Hadoop file system. So you don't really need to track last access time for your files. Disable that for your mount points using "noatime". This will stop linux from keeping track of last access times which is not used anyway.

If you could, in your BIOS settings, change CPU and CPU frequency governance to performance mode. This is a tradeoff between power and performance. So make sure you know what you are doing. your current loads may not already be CPU bound so there may not be any point in doing this. Same things with power settings. you change those to performance for power/performance tradeoff. These are more meant to be sucking juice out of your hardware after you have done everything at OS level, so ideally you don't want to go this far.

[#5 · Accepted solution]

@SBandaru

Please find the below linkto do more tunning if it's help you 🙂

http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-1/

http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-2/

If this is helps plz let us know..

[#6]

@Ashnee Sharma

Perfect, I have done all these in prior. Are they anything left?

[#7]

Please refer for kernel tunning :-

https://tweaked.io/guide/kernel/

https://mapredit.blogspot.in/2014/11/hadoop-server-performance-tuning.html

I think this setting's are enough.

[#8]

If this is the your answer, then lets close this.