Rolling upgrade to 2.6.5

[#1]

hi all

we are in the middle of rolling upgrade

we install the HDP 2.6.5 pkgs from ambari

now we perform rolling upgrade

rolling upgrade window say that spark thrift servers are down and in maintenance mode

this is right we stop the spark thrift services since we are not use the SPARK2_THRIFTSERVER , and not want to do spark upgrade

so only spark clients and spark history are up and running

but rolling upgrade cant continue because ( according to the rolling upgrade message we must to start the spark thrift services and disable the maintenance mode )

so how to continue from this stage?

Reason: The following Service Components should be in a started state.  Please invoke a service Stop and full Start and try again. SPARK2: SPARK2_THRIFTSERVER (in INSTALLED on host master01.sys674.com)
Failed on: SPARK2

[#2 · Accepted solution]

@Michael Bronson

If you are not using Spark2 currently then you should remove the Spark Service prior to starting the HDP upgrade. (one the upgrade is done then you can install it back)

During the upgrade it is not recommended to Add / Remove any service as it can cause severe DB inconsistency.

.

Following link talks about the Prerequisite of HDP upgrade: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-upgrade/content/upgrading_HDP_pre...

Following are the snippet from the above link:

Service Maintenance Mode

No Services can be in Maintenance Mode.

Services Started

All Services must be started.

.

[#3]

@Dear Jay , we need the spark2 , what we not need is the spark thrift servers ,

so we have two choice

1.remove the thrift servers

2.start the thrift serves and perform the upgrade

am I correct ?

[#4]

@Michael Bronson

Yes either option1 or option 2 . If you have not used spark thrift server in your cluster then you can remove it before starting the Upgrade and once the upgrade is done then later whenever you need it then you can just add it by clicking on the individual host page "Add" button.

[#5]

@Dear jay

what I found in add service is only that: ( no thrift component there )

[#6]

@jay so in that case after we remove the thrift component

then how we can bring them back ?

[#7]

@dear jay - I think the only option is to delete the spark2 service and add the spark2 ( by this approach we can create again the thrift component under spark2 )

[#8]

@Michael Bronson

No, you can do this:

Ambari UI --> Spark2 --> Summary Page (here you will see SparkThrift Server 0/2)

Which means it is already installed on 2 host in your cluster.

Just click on the "SparkThrift Server" 0/2 and then it will show the Hosts where it is installed.

Then in the Hosts page of those host you can click on the "Delete" option.

.

[#9]

@jay , yes you right , but maybe I not explain well ,

when you delete the thrift servers ,

then you can add them back

the only option is to delete the spark2 and add spark2 again

please tell me please if I am correct ?

[#10]

@Michael Bronson

You do not need to delete the whole Spark2 Service.

You can delete the "SparkThrift Server" host component which are installed on the two hosts (as per your initial screenshot) and then check if your Spark2 is all working fine ... Prior to the HDP upgrade please check/run the Spark2 service check.

If everything is green and running then you can start upgrading to HDP 2.6.5. Once the upgrade is completed then any time you can click on the

Ambari UI --> Hosts (Tab) --> Specific Host --> "Add" (Button) and then add the "SparkThrift Server" when you need it.