Support Questions
Find answers, ask questions, and share your expertise

Upgrade HDP-3.0.1/HDF-3.2.0 cluster to HDP-3.1.0/HDF-3.3.1: Upgrade paths and experiences?

Upgrade HDP-3.0.1/HDF-3.2.0 cluster to HDP-3.1.0/HDF-3.3.1: Upgrade paths and experiences?

1. question: Is this upgrade supported?

The Doc suggests this is supported (and why shouldn't it), after reading

* table 2 from

* HDF upgrade paths: "HDF 3.2.0" , Supported HDP versions – 3.0.x, 3.1.0

2. question: The exact Upgrade path?

Reference Doc:

The initial upgrade steps from the Documentation are clear to me (that I assume have to be done for both Upgrade cases: on HDP or pure HDF cluster):

* Pre-upgrade tasks: All clear

* Upgrade Ambari and the HDF Management Pack: All clear and done

Then in it is not clear to me

* Upgrading an HDF Cluster : 3.Q: Is it required ? (I assumed this is only required on a pure HDF cluster, though some steps there look they could be useful also on HDP cluster)

* Upgrading HDF 3.2.0 services on an HDP cluster: Required (it exactly reflects my upgrade case)

** Upgrade HDP : As I learned (too late), this step should have been done after initial HDF-Upgrade steps. (Because I had done the HDP upgrade 3.0.1 -> 3.1.0 as a separate Upgrade, leaving HDF services untouched. Big mistake ?!)

*** In step 1.Select the HDP 3.0.x version to which you want to upgrade, and update the base URLs for HDP, HDF, and HDP UTILs. :

4. Question: What if you forget to provide (updated) HDF base URLs in this step? (Any chance to fix this later? I managed to updated/add a new HDF-3.3.1 repo URL later to the existing HDP- stack, but from there Ambari did not offer any way to install/upgrade new HDF packages)

** Upgrade HDF services : clear again

I tried to ask simple, atomic questions, to understand what are my options in the situation I got into:

In fact I did not follow the exact above order of steps (because initially I only read the HDP Upgrade Doc), so my plan was:

* 1.phase (DONE) HDP-3.0.1 (+HDF 3.2.0 services) was upgraded to HDP-3.1.0 (HDF services untouched)

* 2.phase (topic here) HDF Upgrade from 3.2.0 to 3.3.1 on the same cluster

(TODO describe detailed steps I tried. In short I followed the Doc, and did not follow Upgrading an HDF Cluster (see above) , but later no surprise that Upgrade HDF services failed, because no yum repo, or new HDF rpms installed)

New questions arising from that situation:

* 5. Question: What is the upgrade path from HDP 3.2 to HDF 3.3 (3.3.1) on an HDP 3.1 Cluster (that already had HDF 3.2 services installed, and was upgraded from HDP 3.0.1 to HDP 3.1.0) ?

* 6. Question: If my 2-phase upgrade path is not officially supported, is there still a way to get only the HDF 3.2.0 services upgraded?

** 6b. Question: Maybe manually installing the updated HDF repos and the updated rpms (on centos7 with yum install) ? Would this be promising way out? Major concerns?

** 6c. Question: Would first de-installing all HDF services help? (and then again ensure HDF repo updated to 3.3.1, and then follow the documentation 'Install HDF services on existing HDP 3.x cluster')

Thanks in advance!

Any hints welcome (no need to know answers to all questions) :+1:

Maybe @Jay Kumar SenSharma, @Akhil S Naik or @Saurabh ?


(Tomorrow I will merge over my learnings from related question, that was in the wrong *track*: )