- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
All OS settings for RedHat Cluster?
- Labels:
-
Apache Hadoop
Created 07-24-2016 03:01 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I'm planning to install the Ambari 2.1.1 HDP 2.4 on RedHat 7.2 Operating System. Does anyone have a complete checklist of OS settings need to be done before installation? I need something other than this link
Example: vm.swappniess=0 vm.dirty_ratio=0 vm.dirty_background_ratio=0 Changing THP to never
Thanks in advance for providing any information.
Created 07-25-2016 08:02 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@SBandaru
Please find the below linkto do more tunning if it's help you 🙂
http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-1/
http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-2/
If this is helps plz let us know..
Created 07-24-2016 04:08 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think this link is what you are looking for.
Also, I am pretty sure in the newer version of Redhat, you need to setup swappiness to 1 instead of 0. I would also disable transparent_hugepage
do a cat on the the following file on your OS and see if it's set to never or always. If it's at always, change it to never.
/sys/kernel/mm/redhat_transparent_hugepage/defrag
Use the following command to change this value to never
echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
I am not hundred percent sure but I think this requires a restart.
Created 07-24-2016 04:41 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for the quick response, I have already made all the changes which are in that link. Even I have made changes which I mentioned in the example. Now I need more OS level setting which improves cluster performance.
Created 07-24-2016 05:08 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
sure. change swappiness to 1 not 0 which is what you have. Also disable transparent_huge_page as well as transparent_hugepage defrag.
When you are using Hadoop, your files are in Hadoop file system. So you don't really need to track last access time for your files. Disable that for your mount points using "noatime". This will stop linux from keeping track of last access times which is not used anyway.
If you could, in your BIOS settings, change CPU and CPU frequency governance to performance mode. This is a tradeoff between power and performance. So make sure you know what you are doing. your current loads may not already be CPU bound so there may not be any point in doing this. Same things with power settings. you change those to performance for power/performance tradeoff. These are more meant to be sucking juice out of your hardware after you have done everything at OS level, so ideally you don't want to go this far.
Created 07-25-2016 08:02 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@SBandaru
Please find the below linkto do more tunning if it's help you 🙂
http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-1/
http://crazyadmins.com/tune-hadoop-cluster-to-get-maximum-performance-part-2/
If this is helps plz let us know..
Created 07-26-2016 09:06 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Perfect, I have done all these in prior. Are they anything left?
Created 07-27-2016 09:48 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Please refer for kernel tunning :-
https://tweaked.io/guide/kernel/
https://mapredit.blogspot.in/2014/11/hadoop-server-performance-tuning.html
I think this setting's are enough.
Created 08-03-2016 06:30 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If this is the your answer, then lets close this.