Upgrade or Fresh install

lenu — Fri, 16 Sep 2022 13:46:21 GMT

Hi All,

We have 10 node cluster. We have only few teams are using the cluster at the moment.What do you suggest fresh install or Upgrade . If so why ? Could you please explain the pain points of both . What is the best practice here .

Clean installation of OS, HDP , HDF or upgrade of HDP and HDF . If fresh install . we will take a back of all the data to another machine and reinstall everything .

Re: Upgrade or Fresh install

Shelton — Thu, 04 Oct 2018 04:01:34 GMT

@Lenu K

Your question is rather wide for a small cluster all depends on manpower at hand, for HDF remember to back up the flow files, below are immediately what comes into my mind.

Fresh Install pros and con's

Better planned
Here you get a clean installation maybe properly configured mistakes learned from the current cluster setup.
Straightforward no upgrade surprises.
Loose Customization

Upgrade pros and cons'

Must plan properly and document steps
Expect technical surprises and challenge.
Plan support if not having one already on the D-day
Challenges mold you to a better hadoopist!
See Mandatory Post-Upgrade Tasks

Best practice

Verify that the file system you selected is supported HWX
Pre-create all the databases
Backup your cluster before either of the above.
Plan for at least NN/RM HA (NN are the brain so allocate good memory) MUST have 3 Zookeeper
HDD planning is important SSD for SCSI
Restrict access to the cluster from the ONLY edge node.
Kerberize the Cluster
Configure SSL think of SSD for Zk,Hbase and OS can also use the SSD acceleration for temp tables in hive, exposing the SSD via HDFS
Plan well the Data center network(Backup lines)
Size your nodes memory and storage properly.
Beware if performance is a must especially with Kafka and Storm are memory intensive.
Delegate authorization to Ranger.
Test upgrade procedures for new versions of existing components
Execute performance tests of custom-built applications
Allow end-users to perform user acceptance testing
Execute integration tests where custom-built applications communicate with third-party software
Experiment with new software that is beta quality and may not be ready for usage at all
Execute security penetration tests (typically done by an external company)
Let application developers modify configuration parameters and restart services on short notice
Maintain a mirror image of the production environment to be activated in case of natural disaster or unforeseen events
Execute regression tests that compare the outputs of new application code with existing code running in production

HTH

question Re: Upgrade or Fresh install in Support Questions

Upgrade or Fresh install

Re: Upgrade or Fresh install