Installing multiple versions of HDP on the same cluster

[#1]

I am trying to install HDP 2.6.5 on a 2 node system that i have for testing purposes. But I just recently noticed that the HDP versiono 3.0 is also available although with some components removed (Mahout, slider, flume, falcon). After a while, I would like to install HDP 3.0 on the same set of machines without removing the older one as I might still need to use them. I would like to know if this is possible, installing 2 or more versions of HDP on the same cluster, without any issues or is this unwise?

[#2]

@Nanda Kumar

That can't be possible, because when you upgrade all the executables will point to the new version. HDP 3.0 removed some components as you noticed the Ambari views inclusive but there are plans to replace ambari views maybe in tech preview in the new release.

I would advise you have a separate version of HDP clusters, especially if you have the last release of 2.6.x the support still runs and HDP 3.0 is still new so most production clusters will still run 2.6.
There are many changes in HDP 3.0 like namenode federation etc which will need time to adopt.

HTH

[#3]

@Geoffrey Shelton Okot

Thank you. I probably did not specify one point clearly. I'll try to rephrase my question. I want to know if I can have 2 versions of existing in parallel and by that I do not mean upgrade. I want to know if I can install 2.6.5 for now and later in the future another fresh installation on 3.0.0 on the same cluster even if it means that I cannot have the services for both versions running in parallel (say to avoid port conflicts, resource constraint etc). Is this possible? I see that the OS/Java or other requirements to be more or less the same so far for both versions.

[#4]

@Nanda Kumar
To my understanding, there is no option of like Dual boot in an HDP installation where you either start the 2.6.x or the 3.0. As reiterated the only option is to have 2 clusters running separately and putting a Knox gateway in front so you can connect to the cluster of your choice. I haven't tested this setup as yet but looks a more logical approach.

HTH

[#5]

Thank you. However, the 2 installations(2 versions) I am talking about are not connected in any way. I am not looking for a solution with a central management for two or more versions, if that is what my queries have led you to believe. Each of the installations is independent of the other. I will run the version that I prefer but at any time, I will only run one version. I will provide different directories(in the ambari configuration for data, logs etc, not the HDP libraries which would reside in their respective default locations) for each version. With this idea, are there any issues that might arise? The possible issue I can think of is the overwriting of files in the system directories in linux where commands like hadoop, hbase, mapreduce etc from the older installation may get overwritten by the newer one. Do you think there may be other issues too and any plausible solution for them?

[#6]

The solution here is to use 2 nodes for 2.6 and 2 more nodes for 3.0. 2 separate clusters. It is not possible to have 2 versions running on same 2 nodes nor any manner to swap between two. That is just not how this works.

[#7]

Understood. Thank you.

[#8 · Accepted solution]

@Nanda Kumar

Personally, I have not come across an installation with 2 versions on the same host. Yes, files will be overwritten by the newer versions. I advise you to run the different version in Oracle VM, that gives you a physical separation ,economic and yet multi-tenant.
HTH