Upgrade Path

[#1]

Hi @Shelton  and experts,

My Current environment details are:

Ambari version = 2.6.1 running on SLES SP3
HDF version = 3.0.2.4 running on SLES SP3
Apache Kafka = 0.10.2 running on SLES SP3

My Goal is to get Apache Kafka Target version to 2.0.0. As per release notes of HDF version 3.3.0 comes with apache kafka version 2.0.0

To acheive this:

1) what are my upgrade paths for both AMBARI <version ?> and HDF 3.3.0 ?
2) What is the correct sequence of upgrade that I need to follow considering HDF management pack version, Ambari version and HDF version ?

As per support matrix, it says, HDF 3.2.0 and later versions only supports SLES 12 SP3. But in the past we had SLES SP1 OS running (2 years back) so at that time we had installed HDF 3.0.2 and Ambari 2.6.1
Per my research i arrived to following sequence and upgrade path. Could you please validate if this is right upgrade path or not ? If not, please suggest correct path and sequence

Sequence and upgrade path:
===========================
Step 1) Upgrade "HDF Management Pack" to HDF 3.1.0 from HDF 3.0.2

Step 2) Upgrade "HDF" to 3.1.0 (with SLES 12 SP3, support matrix says SP3 not supported) from HDF 3.0.2 (running already on SLES 12 SP3)

Note: The reason for HDF 3.1.0 is document this upgrade path says -> https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.4.0/ambari-managed-hdf-upgrade/content/hdf-upgrade...
"If you are running an earlier HDF version, upgrade to at lease HDF 3.1.0, and then proceed to the HDF 3.3.0 upgrade."

Step 3) Upgrade "Ambari" to 2.7.3 (with SLES 12 SP3) from Ambari 2.6.1 (running already on SLES 12 SP3)

Step 4) Upgrade "HDF Management Pack" to HDF 3.3.0 from HDF 3.1.0

Step 5) Upgrade "HDF" to 3.3.0 (with SLES 12 SP3) from HDF 3.1.0 (with SLES 12 SP3)

Additionally, to get Apache Kafka version 2.0.0 and higher version do we have any other option ? like installing new AMBARI 2.7.4 and Add Kafka using "Add service" option from Ambari ?
If this is the case then what version of Kafka comes allong with AMBARI 2.7.4 new installation ?

Thanks

[#2]

@mhchethan  Referencing your other post, yes, you can get to Kafka 2.0 with Ambari 2.7.4 on HDP 3.1.4.   You can also get the version of kafka you want into HDP cluster using the HDF management packs and the upgrade path you already mentioned to get to current HDF.

The other options for older versions of HDP/HDF:

1. You can make you own management packs.  Unpack the content of an existing management pack, and modify the files/folders to fit the versions you want, then repack and test.   
2. You can also deliver single services you want manually as a Custom Ambari Service.  You basically look at existing service's fileset, copy it, modify it, etc then deliver it to /var/lib/ambari-server/resources/common-services and restart ambari.
3. You can also modify the existing HDP/HDF stacks in a similar manner.  For example: /var/lib/ambari-server/resources/stacks/HDP and may do this in coordination with #2 above.

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

Thanks,

Steven @ DFHZ

[#3]

Thanks a lot @stevenmatison 

So with your responses, can you please conclude following:

1) My sequence and upgrade path mentioned is correct and it will work. Right ?

2) Finally, can you please help me by  sharing  me the link to download binaries for HDF 3.1.0 and HDF 3.3.0 for SLES 12 SP3 ?

support matrix always says SLES 12 SP3 is supported with HDF 3.3.0

But if you open release notes of HDF 3.3.0, i dont see any option to download the HDF management pack for SLES 12 SP3.  Release Notes contains link to download for SLES 12 SP1 only

Thanks