question Re: HDP OS Upgrade/Patching Best practices in Archives of Support Questions (Read Only)

HDP OS Upgrade/Patching Best practices

awatson — Fri, 16 Sep 2022 09:50:49 GMT

Are there any best practices/ documentation around patching or upgrading an OS (e.g. upgrading CentOS 6 --> 7 or security patching ) while the cluster is running?

Thanks,

Re: HDP OS Upgrade/Patching Best practices

bsaini — Tue, 01 Dec 2015 09:12:22 GMT

This post by Lester Martin sums up really well - https://martin.atlassian.net/wiki/pages/viewpage.action?pageId=36044812

Here is a summary -

Since most OS patching / upgrade requires reboot, it is best to schedule such an activity around a scheduled outage.
It is also recommended to go through the exercise in a lower level environment prior to applying changes in a PROD environment.
In order to be apply the changes while the cluster is up, the patch/upgrade will have to be applied in rolling manner by first stopping the components in the host from Ambari, then applying changes, rebooting host and then starting the Hadoop services from Ambari. Repeat for each host.

This process will have to be scripted for a large size cluster. The steps of stopping and starting the cluster can be performed by uzing Ambari APIs.

Re: HDP OS Upgrade/Patching Best practices

nsabharwal — Tue, 01 Dec 2015 09:23:26 GMT

@Andrew Watson This old guide but good hints http://docs.hortonworks.com/HDPDocuments/Ambari-1.6.0.0/bk_upgrading_Ambari/content/ambari-chap8.html

OS upgrade (example)

Re: HDP OS Upgrade/Patching Best practices

awatson — Tue, 01 Dec 2015 23:22:29 GMT

Thanks @bsaini . Do you know how HDFS rebalancing would work during an OS upgrade/rebalance? When would HDFS start trying to rebalance the data residing on the DataNode being bounced?

Re: HDP OS Upgrade/Patching Best practices

prasad4u_com — Tue, 09 Feb 2016 23:19:51 GMT

Hi Neeraj.. Is there any latest hortonworks documentation to know step by step procedure to perform O.S upgrade for hadoop cluster?

Re: HDP OS Upgrade/Patching Best practices

aervits — Tue, 09 Feb 2016 23:22:36 GMT

here's an old one, can't find new https://ambari.apache.org/1.2.1/installing-hadoop-using-ambari/content/ambari-chap8.html

Re: HDP OS Upgrade/Patching Best practices

prasad4u_com — Tue, 09 Feb 2016 23:22:41 GMT

We enabled HA for Namenode and ResourceManager. What are the actions we need to take care while doing O.S updates on HDP cluster?

Re: HDP OS Upgrade/Patching Best practices

aervits — Tue, 09 Feb 2016 23:24:02 GMT

go through each server one by one, put services in maintenance mode on that host and stop them. Upgrade OS to latest minor version, reboot if necessary and restart services. Go to the next. @Ram D

Re: HDP OS Upgrade/Patching Best practices

prasad4u_com — Tue, 09 Feb 2016 23:32:20 GMT

Thank you @Artem Ervits

Re: HDP OS Upgrade/Patching Best practices

clvi — Wed, 01 Jun 2016 19:20:46 GMT

Thx @bsani

We have sat HA on our name nodes while we don't want the cluster to be unawailable.

So is there a best practice for doing patching on a cluster that is supposed to be available 24/7.

How to avoid rebalancing during patching

When upgrading datanodes in chuncks, is there a way to make sure that replica of a data block is available on one of the servers alive

/Claus