Impact of OS upgrade from RHEL 6-->RHEL7 for hadoop cluster.

[#1]

Hi All,

We are planning to upgrade our Hadoop cluster, so before that we need to upgrade OS from RHEL 6.8-->RHEL7.x, by doing this OS level upgrade whether there will be any impact for HDP and Ambari level.or Upgrading the OS level will leads to Re-clustering of hadoop. kindly suggest the better way to Upgrade OS and Hadoop cluster and if the OS upgrade happens does it impact Hadoop cluster.

Regards,

Mathi

[#2]

@Mathi Murugan

There is no standardised steps or pre-requisites defined for HDP cluster during OS upgrade. The release bits come from different repositories for Ambari and HDP, so will need to be replaced.

A high level overview of the process is something like:

1. Check compatibility of all components for the new OS (e.g. will java, python etc be replaced? will HUE still work? Is our ODBC driver available?)
2. Ensure everything is backed up.
3. Cleanly shutdown the cluster (including savenamespace)
4. Upgrade the OS according to their vendors advice
5. Install the relevant HDP bits for their new OS
6. Startup Ambari and update the repository for the new OS
7. Startup HDP.

[#3]

This is not a viable answer when you have scores of hosts (over 80 and growing) involved. you are talking hours and hours of downtime. That simply is not realistic in a production (let alone a testing) cluster.

[#4]

@Mathi Murugan, I am looking at the same thing. There should be NO REASON for this to be difficult. Each set of hosts with different OS's (as long as part of an effort to upgrade the base OS for the respective hosts) would have access to their own repositories should not be so complex to apply. But Alas, this is. There are many inquiries about this very issue and all are simply not simple, no pun intended.

This is supposed to be an open system. Logically, this makes no sense to not only adding new hosts to an existing cluster with a newer OS, but it should also be a better process to upgrade in place. Maybe I am expecting an unattainable outcome, not sure. But there has to be a clear path to address OS upgrades for large scale deployments

[#5]

@Mathi Murugan There is support for multiple OSes in Ambari as of Ambari 2.5.2. More information can be found in this guide on how to upgrade OS to a new version.

[#6]

Yes! After working through product support, we figured out that Ambari 2.5.2 is the key to transitioning to upgraded OS. In our case RH 7.2.

[#7]

Could you share the details of you HDP upgrade? I'm suing Ambari to manage HDF (Nifi) and am having a few issues getting the services (zookeeper at this point) to come up