Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDP Express Upgrade to 3.1.0.0 halts / fails at Update Target Repositories

Highlighted

HDP Express Upgrade to 3.1.0.0 halts / fails at Update Target Repositories

New Contributor

I am currently doing an upgrade from HDP-2.6.0.3 to HDP-3.1.0.0 on a HDP Cluster

Previously I installed

Unfortunately did not read upgrade Guide for HDF before staring the upgrade.

For the upgrade process I upgraded ambari to version 2.7.3.0 (without touching the mpacks)

Before starting the HDP upgrade there was the warning that MongoDB and Nifi would be uninstalled because they don't belong to the HDP stack, which was fine for me.

The HDP Upgrade, which I started after the ambari upgrade at first halted / failed in the step Remove unsupported services and components due to MongoDB and HDF. I paused the upgrade and uninstalled the MongoDB and Nifi Services through ambari as well as uninstalling the mongodb mpack and upgrading the hdp mpack to version 3.4.0.0-155.

Then the HDP upgrade proceeded but failed at Update Target Repositories with a stdout of

Updating the desired repository version to 3.1.0.0-78 for all cluster services.

and a stderr of

org.apache.ambari.server.ServiceNotFoundException: Service not found, clusterName=itpp, serviceName=MONGODB
    at org.apache.ambari.server.state.cluster.ClusterImpl.getService(ClusterImpl.java:888)
    at org.apache.ambari.server.state.UpgradeHelper.setDesiredRepositories(UpgradeHelper.java:922)
    at org.apache.ambari.server.state.UpgradeHelper.updateDesiredRepositoriesAndConfigs(UpgradeHelper.java:894)
    at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:118)
    at org.apache.ambari.server.serveraction.upgrades.UpdateDesiredRepositoryAction.updateDesiredRepositoryVersion(UpdateDesiredRepositoryAction.java:160)
    at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:128)
    at org.apache.ambari.server.serveraction.upgrades.UpdateDesiredRepositoryAction.execute(UpdateDesiredRepositoryAction.java:96)
    at org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.execute(ServerActionExecutor.java:550)
    at org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.run(ServerActionExecutor.java:466)
at java.lang.Thread.run(Thread.java:745)

I checked

https://community.hortonworks.com/questions/202455/upgrade-from-hdp-264-to-300-fails-at-update-targe...

but unfortunately the answers provided there where either not helpful for me or missing some details. I did check for MONGODB and NIFI in the tables

servicecomponentdesiredstate
hostcomponentstate
hostcomponentdesiredstate
servicedesiredstate
clusterservices

and did not find any related entries. It seems that there are still some references to Nifi and/or Mongodb somewhere the in ambari database.

In the ambari-server.log there is also the error:

Failed to obtain service info for stack HDP-2.6, service name NIFI
org.apache.ambari.server.StackAccessException: Stack data, stackName=HDP, stackVersion=2.6, serviceName=NIFI

Any advice how to proceed or how to restart the upgrade (if advisable)?

Don't have an account?
Coming from Hortonworks? Activate your account here