question Re: 4 Required subjects before upgrade in Support Questions

4 Required subjects before upgrade

mike_bronson7 — Sun, 18 Aug 2019 07:43:02 GMT

we are in the stage before the upgrade , ( but we see 4 requirements that need to be solve )

message is - You must meet these requirements before you can proceed.

example:

but the requirement not really true

we check the services and we not found any service that is in maintenance mode

second all services in the clsuter are started and up

and nothing that described in requirements is true

so my question is

what we can do in this stage ?

we cant do upgrade! ,

and actually we cant uninstall because we not see any option to uninstall !!!

please advice regarding this case

Re: 4 Required subjects before upgrade

Shelton — Mon, 19 Mar 2018 00:42:30 GMT

@Michael Bronson

Can you share the Ambari UI screenshot of the cluster to be upgraded?
Those are basic pre-requisites

Re: 4 Required subjects before upgrade

mike_bronson7 — Mon, 19 Mar 2018 00:48:00 GMT

@Geoffrey Shelton Okot

for this time I cant because securty aspect , but I can tell you for sure that all services are up

Re: 4 Required subjects before upgrade

mike_bronson7 — Mon, 19 Mar 2018 00:49:25 GMT

second , why we cant uninstall ( this option not defined in ambari GUI )

Re: 4 Required subjects before upgrade

Shelton — Mon, 19 Mar 2018 01:06:14 GMT

@Michael Bronson

Visually I can't tell at what stage you'd reached with the upgrade. What is the version of the current HDP cluster, I could try to reproduce it if I have a similar version?

Have you tried restarting all the ambari-agents maybe it lost connection while pooling because that looks a queer message if the your components are running and not in maintenance mode 🙂

Re: 4 Required subjects before upgrade

mike_bronson7 — Mon, 19 Mar 2018 01:10:50 GMT

the current version is HDP-2.6.0.3 and the target is to upgrade the version to 2.6.4.0 , yes we restart all ambari agents and ambari server but this action not help , ( and no any service that run in mantence mode , we check that twice )

Re: 4 Required subjects before upgrade

Shelton — Mon, 19 Mar 2018 01:21:12 GMT

@Michael Bronson

You forgot the source HDP version ... 🙂

Re: 4 Required subjects before upgrade

mike_bronson7 — Mon, 19 Mar 2018 01:23:13 GMT

sorry - HDP-2.6.0.3

Re: 4 Required subjects before upgrade

mike_bronson7 — Mon, 19 Mar 2018 01:27:01 GMT

@Geoffrey Shelton Okot is it possible to do uninstall , as was before the install?

Re: 4 Required subjects before upgrade

mike_bronson7 — Mon, 19 Mar 2018 01:41:02 GMT

another interning thing we see that repo is :

why - repo-52-repo-54 ? , even yum clean all not help!


 more ambari-hdp-54.repo
[HDP-2.6-repo-52-repo-54]
name=HDP-2.6-repo-52-repo-54
baseurl=http://master02.sys65.com/HDP/centos7/2.6.4.0-91
path=/
enabled=1
gpgcheck=0
[HDP-UTILS-1.1.0.22-repo-52-repo-54]
name=HDP-UTILS-1.1.0.22-repo-52-repo-54
baseurl=http://master02.sys65.com
path=/
enabled=1
gpgcheck=0

Re: 4 Required subjects before upgrade

Shelton — Mon, 19 Mar 2018 01:45:06 GMT

@Michael Bronson

My current version is HDP-2.6.2.0 so I am upgrading to HDP-2.6.3.0 then will do the 2.6.4.0-91 .

Is that a production cluster? I think until you finalize(the moment symlinks are created to point to the new version) your previous version should still be usuable. !!

Re: 4 Required subjects before upgrade

mike_bronson7 — Mon, 19 Mar 2018 02:34:55 GMT

yes my cluster is prod cluster

Re: 4 Required subjects before upgrade

mike_bronson7 — Mon, 19 Mar 2018 02:43:32 GMT

before the HDP Upgrade ( install ) we finalize the current version , so this isnt the problem

Re: 4 Required subjects before upgrade

mike_bronson7 — Mon, 19 Mar 2018 02:45:04 GMT

the problem is that we cant upgrade from this point ( because the 4 requirement ) and we also cant falback

Re: 4 Required subjects before upgrade

Shelton — Mon, 19 Mar 2018 03:34:05 GMT

@Michael Bronson

If its a production cluster then by precaution I advise you to contact hortonworks support hoping you subscribed. Secondly before upgrading the production you should have tested in DEV, UAT and maybe PREPROD and clearly documented all issues encountered and it's resolution, that's the safest and recommended path.

You will have to resolve those issues before you can proceed, but as reiterated, before you finalize your upgrade your old version should still be usable as all the conf files still are pointing to HDP-2.6.0.3.

To this

$ cd /usr/hdp/current
$ ll
lrwxrwxrwx 1 root root 26 Mar 17 18:39 sqoop-client -> /usr/hdp/2.6.2.0-205/sqoop
lrwxrwxrwx 1 root root 26 Mar 17 18:39 sqoop-server -> /usr/hdp/2.6.2.0-205/sqoop
lrwxrwxrwx 1 root root 26 Mar 17 18:39 storm-client -> /usr/hdp/2.6.2.0-205/storm
lrwxrwxrwx 1 root root 26 Mar 17 18:39 storm-nimbus -> /usr/hdp/2.6.2.0-205/storm
lrwxrwxrwx 1 root root 40 Mar 17 18:39 storm-slider-client -> /usr/hdp/2.6.2.0-205/storm-slider-client

This should still point to 2.6.0.3

I am still upgrading to an intermediary version

Re: 4 Required subjects before upgrade

mike_bronson7 — Mon, 19 Mar 2018 03:42:12 GMT

you said " My current version is HDP-2.6.2.0 so I am upgrading to HDP-2.6.3.0 then will do the 2.6.4.0-91"

why not upgrade from HDP-2.6.2.0 to 2.6.4.0-91 , why need middle version ?

Re: 4 Required subjects before upgrade

Shelton — Mon, 19 Mar 2018 05:57:48 GMT

@Michael Bronson

Wanted t have the same path like you

Re: 4 Required subjects before upgrade

jsensharma — Mon, 19 Mar 2018 15:25:16 GMT

@Michael Bronson
I see 4 messages in your Initial Issue description.

Message-1).

Verify Cluster Components Exist In Target Repository

This kind of message can appear while starting upgrades if we have installed some custom services to our cluster which are not part of the HDP version that we are going to install. For example if you have some third party services installed in your cluster which are not coming from HDP repo then you will have to remove them before you begin the HDP upgrade.

Message-2).

All service components must be installed

This messages indicates that you might have some services installed to your cluster which migth have some components in "INSTALL_FAILED" state.

# curl -u admin:admin -H "X-Requested-By: ambari" -X GET http://hdfcluster1.example.com:8080/api/v1/clusters/TestCluster/host_components?HostRoles/state=INSTALL_FAILED

If you find any component in install failed state then please install them properly before starting the upgrade.

In some cases it might happen if the clients components might not be installed on few cluster hosts. So please check if you have CLIENT components installed properly on your cluster nodes. If not then use the following API aapproach to install the CLIENTs to your cluster nodes. (or install them using Ambari UI)

https://community.hortonworks.com/content/supportkb/150098/how-to-install-hadoop-clients-on-all-the-hosts-in.html

Message-3).

No Service can be in maintenance mode.

You can not start HDP upgrade if any of your service is in Maintenance mode so please run the following API call before upgrading HDp to findout if there are any Services in Maintenance Mode or not? Also similar approach you will need to use to check that all Host Components are also not in Maintenance Mode .

# curl -u admin:admin -H "X-Requested-By: ambari" -X GET http://hdfcluster1.example.com:8080/api/v1/clusters/TestCluster/services?ServiceInfo/maintenance_state=ON

Message-4).

All Services must be started.

This message indicates that we can not start HDP component upgrades before all the components are already in running state. So we need to make sure that all the components / services are in Started stated. Also please make sure that we must run all the service checks successfully before we begin the HDP component upgrade to verify that all the components are in healthy state..

Re: 4 Required subjects before upgrade

mike_bronson7 — Mon, 19 Mar 2018 15:55:02 GMT

regarding to steps 2,3 very strange because all services/components are without failed and no service / components with maintenance mode

<br>[root@master02 centos7]# curl -u admin:admin -H "X-Requested-By: ambari" -X GET http://master02:8080/api/v1/clusters/sys65/host_components?HostRoles/state=INSTALL_FAILED                    {
  "href" : "http://master02:8080/api/v1/clusters/sys65/host_components?HostRoles/state=INSTALL_FAILED",
  "items" : [ ]


}[root@master02 centos7]#curl -u admin:admin -H "X-Requested-By: ambari" -X GET http://master02:8080/api/v1/clusters/sys65/services?ServiceInfo/maintenance_state=ON
{
  "href" : "http://master02:8080/api/v1/clusters/sys65/services?ServiceInfo/maintenance_state=ON",
  "items" : [ ]

Re: 4 Required subjects before upgrade

mike_bronson7 — Mon, 19 Mar 2018 16:04:13 GMT

regarding to step 4 - we check more then twice that all service are up and running