Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How can we do O.S updates in the real time Prod cluster? Can anyone tell me the procedure?

How can we do O.S updates in the real time Prod cluster? Can anyone tell me the procedure?

Expert Contributor
 
8 REPLIES 8
Highlighted

Re: How can we do O.S updates in the real time Prod cluster? Can anyone tell me the procedure?

@Ram D

What type of update is this? Would you have downtime for this? How many nodes are you looking at upgrading? If this is a small patch update then you could target to do a set of Slaves at a time first. Then if you have HA enabled for key services like Namenode /Hiveserver2, you could upgrade Masters one at a time when the other Master in HA is up and running.

Highlighted

Re: How can we do O.S updates in the real time Prod cluster? Can anyone tell me the procedure?

Expert Contributor

@vpoornalingam.

We are having a cluster of 30 node, we are planning to do cent o.s updates on this cluster. We have HA enabled for Namenode and ResourceManager. What is the best procedure to follow? We are having 18 data nodes, 2 name nodes, 2 resourcemanagers, 3 journal nodes. Can you please suggest me the flow in detail?

Highlighted

Re: How can we do O.S updates in the real time Prod cluster? Can anyone tell me the procedure?

Expert Contributor

@Artem Ervits

Hey , i am unable to see your post. Can you please share your views too? Thank you in advance.

Highlighted

Re: How can we do O.S updates in the real time Prod cluster? Can anyone tell me the procedure?

Mentor

@Ram D I decided to delete it as it's basically a repeat of what @vpoornalingam suggested. What I suggested was for small clusters, schedule a downtime with your internal customers. For your cluster with 30 nodes, doing patches on slaves one by one with maintenance mode on, stopping components, patching, retart OS, restart components, remove maintenance mode. Then do the same with masters one at a time in the same order of events.

Highlighted

Re: How can we do O.S updates in the real time Prod cluster? Can anyone tell me the procedure?

Expert Contributor

Thank you @Artem Ervits

Highlighted

Re: How can we do O.S updates in the real time Prod cluster? Can anyone tell me the procedure?

Mentor

sure thing, keep the questions coming :) @Ram D

Highlighted

Re: How can we do O.S updates in the real time Prod cluster? Can anyone tell me the procedure?

Expert Contributor

@vpoornalingam

I just tried to stop the service through REST api after maintenance mode on, it was not succeeded. Can you please check below link?

When Maintenance mode is ON:

The following behavior also results:

  • Alerts are suppressed for the DataNode.
  • DataNode is skipped from HDFS Start/Stop/Restart All, Rolling Restart.
  • DataNode is skipped from all Bulk Operations except Turn Maintenance Mode ON/OFF.
  • DataNode is skipped from Start All and / Stop All components.
  • DataNode is skipped from a host-level restart/restart all/stop all/start.

http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.1.0/bk_Ambari_Users_Guide/content/_maintenance_m...

For O.S updates,

If we put the host in maintenance mode, all the components on host will be in maintenance mode. May i need to stop the services on that host or not? Its not allowing me to stop.

If i do the process like below, its getting worked with REST apis

Stop the services--> turn on maintenance mode --> turn off maintenance mode---> start the services.

Please let me know the process?

Highlighted

Re: How can we do O.S updates in the real time Prod cluster? Can anyone tell me the procedure?

Expert Contributor

@Artem Ervits Can you please check the process what i posted? Please suggest me the flow?

Don't have an account?
Coming from Hortonworks? Activate your account here