Question on HDF upgrade to 3.4.0 Version from HDF 3.0.2

[#1]

My requirement is to upgrade HDF (not HDP) . Basically my environment have two separate clusters - HDP and HDF

I want to upgrade only HDF to 3.4.0 Version from HDF 3.0.2 . Ambari version currently installed is 2.6.1 and HDF version is 3.0.2 . Can you please help me to know if I can directly upgrade to 3.4.0 from 3.0.2 ?

I tried looking into this but no luck

https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.4.0/ambari-managed-hdf-upgrade/content/hdf-upgr...

Can i directly perform HDF upgrade to 3.4.0 from 3.0.2.5 ? Or does it requires first to upgrade to some version first and then upgrade it to higher version hdf 3.4.0 ?

[#2 · Accepted solution]

@chethan mh

The same HWX document categorically states "If you are running an earlier HDF version, upgrade to at lease HDF 3.1.0, and then proceed to the HDF 3.3.0 upgrade." only HDF 3.3.x and HDF 3.2.x can be directly upgraded to 3.4.0.

So you will have a 3 step migration first to at least 3.1.0 then HDF 3.3.0 then finally to 3.4.0. If the effort the try export NiFi can export/import flows via templates. You can save your flow as a template (xml file), and import the template from a file as well.

If you want to save the entire flow you have in the system, you can also find that in nifi/conf/flow.xml.gz on your nifi box. This is not a template but would be able to drop into a clean NiFi instance.

Follow this link to nifi procedure

Please revert

[#3]

Thanks for your reply

To arrive HDF 3.4.0, we also have to upgrade our Ambari as well. Basically we have two separate HDF and HDP clusters. Currently we are planning to upgrade only HDF cluster. We will not touch HDP cluster

Current HDF version = 3.0.2
Current Ambari version = 2.6.1

So, my question is, what should be my upgrade sequence since I need to upgrade Ambari as well along with HDF ?

Note, HDF version 3.1.0 is NOT compatible with Ambari version 2.7.3. Can you recommend which of the below sequence will be good to follow ?

Sequence#1:

Step 1) Upgrade first HDF to 3.1.0 from 3.0.2
This is because HDF 3.1.0 is compatible with Ambari 2.6.1
At the end of this upgrade you will have HDF version 3.1.0 and Ambari version 2.6.1
The first step in upgrading the HDF 3.1 is to upgrade to Ambari 2.6.1.

Step 2) Upgrade Ambari alone to 2.7.3 from 2.6.1
At the end of this upgrade you will have HDF version 3.1.0 and Ambari version 2.7.3

Ref - https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.3.0/ambari-managed-hdf-upgrade/content/hdf-upgr...

Note: HDF version 3.1.0 is NOT compatible with Ambari version 2.7.3

Step 3) Upgrade “HDF” component to 3.3.0 from 3.1.0
At the end of this upgrade you will have HDF version 3.3.0 and Ambari version 2.7.3
HDF version 3.3.0 is compatible with Ambari version 2.7.3

Step 4) Upgrade “HDF” component to 3.4.0 from 3.3.0
At the end of this upgrade you will have HDF version 3.4.0 and Ambari version 2.7.3
HDF version 3.4.0 is compatible with Ambari version 2.7.3

OR

Sequence#2:

Step 1) Upgrade Ambari alone to 2.7.3 from 2.6.1
At the end of this upgrade you will have HDF version 3.0.2 and Ambari version 2.7.3

Note: HDF version 3.0.2 is NOT compatible with Ambari version 2.7.3

Step 2) Upgrade HDF to 3.1.0 from 3.0.2
At the end of this upgrade you will have HDF version 3.1.0 and Ambari version 2.7.3

Note: HDF version 3.1.0 is NOT compatible with Ambari version 2.7.3


Step 3) Upgrade “HDF” component to 3.3.0 from 3.1.0
At the end of this upgrade you will have HDF version 3.3.0 and Ambari version 2.7.3
HDF version 3.3.0 is compatible with Ambari version 2.7.3


Step 4) Upgrade “HDF” component to 3.4.0 from 3.3.0
At the end of this upgrade you will have HDF version 3.4.0 and Ambari version 2.7.3
HDF version 3.4.0 is compatible with Ambari version 2.7.3

Thanks

Chethan

[#4]

Thanks for your reply

To arrive HDF 3.4.0, we also have to upgrade our Ambari as well. Basically we have two separate HDF and HDP clusters. Currently we are planning to upgrade only HDF cluster. We will not touch HDP cluster

Current HDF version = 3.0.2

Current Ambari version = 2.6.1

So, my question is, what should be my upgrade sequence since I need to upgrade Ambari as well along with HDF ?

Note, HDF version 3.1.0 is NOT compatible with Ambari version 2.7.3. Can you recommend which of the below sequence will be good to follow ?

Sequence#1:

Step 1) Upgrade first HDF to 3.1.0 from 3.0.2

This is because HDF 3.1.0 is compatible with Ambari 2.6.1

At the end of this upgrade you will have HDF version 3.1.0 and Ambari version 2.6.1

The first step in upgrading the HDF 3.1 is to upgrade to Ambari 2.6.1.

Step 2) Upgrade Ambari alone to 2.7.3 from 2.6.1

At the end of this upgrade you will have HDF version 3.1.0 and Ambari version 2.7.3

Ref - https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.3.0/ambari-managed-hdf-upgrade/content/hdf-upgr...

Note: HDF version 3.1.0 is NOT compatible with Ambari version 2.7.3

Step 3) Upgrade “HDF” component to 3.3.0 from 3.1.0

At the end of this upgrade you will have HDF version 3.3.0 and Ambari version 2.7.3

HDF version 3.3.0 is compatible with Ambari version 2.7.3

Step 4) Upgrade “HDF” component to 3.4.0 from 3.3.0

At the end of this upgrade you will have HDF version 3.4.0 and Ambari version 2.7.3

HDF version 3.4.0 is compatible with Ambari version 2.7.3

OR

Sequence#2:

Step 1) Upgrade Ambari alone to 2.7.3 from 2.6.1

At the end of this upgrade you will have HDF version 3.0.2 and Ambari version 2.7.3

Note: HDF version 3.0.2 is NOT compatible with Ambari version 2.7.3

Step 2) Upgrade HDF to 3.1.0 from 3.0.2

At the end of this upgrade you will have HDF version 3.1.0 and Ambari version 2.7.3

Note: HDF version 3.1.0 is NOT compatible with Ambari version 2.7.3

Step 3) Upgrade “HDF” component to 3.3.0 from 3.1.0

At the end of this upgrade you will have HDF version 3.3.0 and Ambari version 2.7.3

HDF version 3.3.0 is compatible with Ambari version 2.7.3

Step 4) Upgrade “HDF” component to 3.4.0 from 3.3.0

At the end of this upgrade you will have HDF version 3.4.0 and Ambari version 2.7.3

HDF version 3.4.0 is compatible with Ambari version 2.7.3

Thanks

Chethan

[#5]

Thanks @Geoffrey Shelton Okot ! This helped

[#6]

Thanks for your reply

To arrive HDF 3.4.0, we also have to upgrade our Ambari as well. Basically we have two separate HDF and HDP clusters. Currently we are planning to upgrade only HDF cluster. We will not touch HDP cluster

Current HDF version = 3.0.2
Current Ambari version = 2.6.1

So, my question is, what should be my upgrade sequence since I need to upgrade Ambari as well along with HDF ?

Note, HDF version 3.1.0 is NOT compatible with Ambari version 2.7.3. Can you recommend which of the below sequence will be good to follow ?

Sequence#1:

Step 1) Upgrade first HDF to 3.1.0 from 3.0.2
This is because HDF 3.1.0 is compatible with Ambari 2.6.1
At the end of this upgrade you will have HDF version 3.1.0 and Ambari version 2.6.1
The first step in upgrading the HDF 3.1 is to upgrade to Ambari 2.6.1.

Step 2) Upgrade Ambari alone to 2.7.3 from 2.6.1
At the end of this upgrade you will have HDF version 3.1.0 and Ambari version 2.7.3

Ref - https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.3.0/ambari-managed-hdf-upgrade/content/hdf-upgr...

Note: HDF version 3.1.0 is NOT compatible with Ambari version 2.7.3

Step 3) Upgrade “HDF” component to 3.3.0 from 3.1.0
At the end of this upgrade you will have HDF version 3.3.0 and Ambari version 2.7.3
HDF version 3.3.0 is compatible with Ambari version 2.7.3

Step 4) Upgrade “HDF” component to 3.4.0 from 3.3.0
At the end of this upgrade you will have HDF version 3.4.0 and Ambari version 2.7.3
HDF version 3.4.0 is compatible with Ambari version 2.7.3

OR

Sequence#2:

Step 1) Upgrade Ambari alone to 2.7.3 from 2.6.1
At the end of this upgrade you will have HDF version 3.0.2 and Ambari version 2.7.3

Note: HDF version 3.0.2 is NOT compatible with Ambari version 2.7.3

Step 2) Upgrade HDF to 3.1.0 from 3.0.2
At the end of this upgrade you will have HDF version 3.1.0 and Ambari version 2.7.3

Note: HDF version 3.1.0 is NOT compatible with Ambari version 2.7.3


Step 3) Upgrade “HDF” component to 3.3.0 from 3.1.0
At the end of this upgrade you will have HDF version 3.3.0 and Ambari version 2.7.3
HDF version 3.3.0 is compatible with Ambari version 2.7.3


Step 4) Upgrade “HDF” component to 3.4.0 from 3.3.0
At the end of this upgrade you will have HDF version 3.4.0 and Ambari version 2.7.3
HDF version 3.4.0 is compatible with Ambari version 2.7.3

Thanks

Chethan

[#7]

Thanks for your reply

To arrive HDF 3.4.0, we also have to upgrade our Ambari as well. Basically we have two separate HDF and HDP clusters. Currently we are planning to upgrade only HDF cluster. We will not touch HDP cluster

Current HDF version = 3.0.2
Current Ambari version = 2.6.1

So, my question is, what should be my upgrade sequence since I need to upgrade Ambari as well along with HDF ?

Note, HDF version 3.1.0 is NOT compatible with Ambari version 2.7.3. Can you recommend which of the below sequence will be good to follow ?

Sequence#1:

Step 1) Upgrade first HDF to 3.1.0 from 3.0.2
This is because HDF 3.1.0 is compatible with Ambari 2.6.1
At the end of this upgrade you will have HDF version 3.1.0 and Ambari version 2.6.1
The first step in upgrading the HDF 3.1 is to upgrade to Ambari 2.6.1.

Step 2) Upgrade Ambari alone to 2.7.3 from 2.6.1
At the end of this upgrade you will have HDF version 3.1.0 and Ambari version 2.7.3

Ref - https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.3.0/ambari-managed-hdf-upgrade/content/hdf-upgr...

Note: HDF version 3.1.0 is NOT compatible with Ambari version 2.7.3

Step 3) Upgrade “HDF” component to 3.3.0 from 3.1.0
At the end of this upgrade you will have HDF version 3.3.0 and Ambari version 2.7.3
HDF version 3.3.0 is compatible with Ambari version 2.7.3

Step 4) Upgrade “HDF” component to 3.4.0 from 3.3.0
At the end of this upgrade you will have HDF version 3.4.0 and Ambari version 2.7.3
HDF version 3.4.0 is compatible with Ambari version 2.7.3

OR

Sequence#2:

Step 1) Upgrade Ambari alone to 2.7.3 from 2.6.1
At the end of this upgrade you will have HDF version 3.0.2 and Ambari version 2.7.3

Note: HDF version 3.0.2 is NOT compatible with Ambari version 2.7.3

Step 2) Upgrade HDF to 3.1.0 from 3.0.2
At the end of this upgrade you will have HDF version 3.1.0 and Ambari version 2.7.3

Note: HDF version 3.1.0 is NOT compatible with Ambari version 2.7.3


Step 3) Upgrade “HDF” component to 3.3.0 from 3.1.0
At the end of this upgrade you will have HDF version 3.3.0 and Ambari version 2.7.3
HDF version 3.3.0 is compatible with Ambari version 2.7.3


Step 4) Upgrade “HDF” component to 3.4.0 from 3.3.0
At the end of this upgrade you will have HDF version 3.4.0 and Ambari version 2.7.3
HDF version 3.4.0 is compatible with Ambari version 2.7.3

Thanks

Chethan