Support Questions
Find answers, ask questions, and share your expertise

HDP OS Upgrade/Patching Best practices

Are there any best practices/ documentation around patching or upgrading an OS (e.g. upgrading CentOS 6 --> 7 or security patching ) while the cluster is running?

Thanks,

1 ACCEPTED SOLUTION

This post by Lester Martin sums up really well - https://martin.atlassian.net/wiki/pages/viewpage.action?pageId=36044812

Here is a summary -

  1. Since most OS patching / upgrade requires reboot, it is best to schedule such an activity around a scheduled outage.
  2. It is also recommended to go through the exercise in a lower level environment prior to applying changes in a PROD environment.
  3. In order to be apply the changes while the cluster is up, the patch/upgrade will have to be applied in rolling manner by first stopping the components in the host from Ambari, then applying changes, rebooting host and then starting the Hadoop services from Ambari. Repeat for each host.

This process will have to be scripted for a large size cluster. The steps of stopping and starting the cluster can be performed by uzing Ambari APIs.

View solution in original post

9 REPLIES 9

This post by Lester Martin sums up really well - https://martin.atlassian.net/wiki/pages/viewpage.action?pageId=36044812

Here is a summary -

  1. Since most OS patching / upgrade requires reboot, it is best to schedule such an activity around a scheduled outage.
  2. It is also recommended to go through the exercise in a lower level environment prior to applying changes in a PROD environment.
  3. In order to be apply the changes while the cluster is up, the patch/upgrade will have to be applied in rolling manner by first stopping the components in the host from Ambari, then applying changes, rebooting host and then starting the Hadoop services from Ambari. Repeat for each host.

This process will have to be scripted for a large size cluster. The steps of stopping and starting the cluster can be performed by uzing Ambari APIs.

Thanks @bsaini . Do you know how HDFS rebalancing would work during an OS upgrade/rebalance? When would HDFS start trying to rebalance the data residing on the DataNode being bounced?

New Contributor

Thx @bsani

We have sat HA on our name nodes while we don't want the cluster to be unawailable.

So is there a best practice for doing patching on a cluster that is supposed to be available 24/7.

How to avoid rebalancing during patching

When upgrading datanodes in chuncks, is there a way to make sure that replica of a data block is available on one of the servers alive

/Claus

Expert Contributor

Hi Neeraj.. Is there any latest hortonworks documentation to know step by step procedure to perform O.S upgrade for hadoop cluster?

Mentor

Expert Contributor

We enabled HA for Namenode and ResourceManager. What are the actions we need to take care while doing O.S updates on HDP cluster?

Mentor

go through each server one by one, put services in maintenance mode on that host and stop them. Upgrade OS to latest minor version, reboot if necessary and restart services. Go to the next. @Ram D

Expert Contributor

Thank you @Artem Ervits

; ;