Register Custom Stack Version in hdp-select

[#1]

Hi,

I'm trying to create a custom stack that extends HDP-3.0.

I've readed all the documentation available and after some weeks of works finally I've packaged a mpack.

The installation process goes fine. But in the version management begins the problems.

The custom stack have 3.0.1.0.1 as Version. So the stack_id is HDP-3.0.1.0.1
When I go in the "Stack and versions" sections I see the extended version installed (in my case HDP-3.0.1.0). This is the version that HDP-3.0.1.0.1 extends.

If I go to "Manage Version" -> "Register Version" I can add HDP-1.0. Then in "Stack and versions" the stack presents "Install packages" button. If I run this command I receive this error:

"Cannot determine actual version installed by checking the delta between versions before and after installing package"

If I go to the server and run "hdp-select versions" I see only this:

3.0.1.0-187

What I'm doing wrong? If the custom stack extends the 3.0, I guess it inherits all the configurations (stack_features, stack_tools) and the hooks.

[#2]

It looks like you are attempting to register a repository where the bits are actually 3.0.1.0-87, yet you expect the repository to report 3.0.1.0.1. That is not how stack inheritance works in Ambari. Yes, you can inherit a stack in order to get its configurations, alerts, scripts, etc. However, your new stack is going to be installed from a yum repository which it defines (not inherits). In this case, you are specifying the HDP-3.0.1.0 repository URLs for the RPMs and that means it will install 3.0.1.0-87.

The version information of a package is used for managing a cluster with respect to upgrades. Unless you are actually rebuilding the binaries with a brand new version, then your 3.0.1.0.1 stack will never install.

The wildcard 3_0_1_0_* also will never work here since it becomes ambiguous which repository you want to install from. So, even if you had rebuilt all of the binaries and provided correct repository URLs, a `yum install zookeeper_3_0_1_0_*` would produce random results.

[#3]

Ok,

that's make sense. So I don't know which approach I have to use.

I'd like to buld a custom distribution that extends HDP-3.0. This distribution consist in a bunch of custom software that requires some components of HDP-3.0 (Zookeeper, Kafka, Hdfs, etc).

I need to make the custom services upgradable, so I've supposed to use the mpacks defining a custom stack.

[#4]

Unless you are going to rebuild those services with a new version number and package them all together, what you are attempting to do won't work. All services from a repository must advertise the same version. The version select tool which is used (say hdp-select) must be able to interact will all of them, which means that it would also need to support your custom services.

There's no way to make custom services upgradeable if they don't actually belong to that stack and advertise the same version.

Take HDP and HDF, for example. When you install either, you'll get ZooKeeper. However, the ZooKeeper which comes with HDP is packaged differently than that which comes with HDF. The repositories used to install HDF will install ZooKeeper from a version which matches the HDF version. This allows the HDF stack to be managed as an upgradable stack since all of the components are built together.

On the other hand, if you just drop an HDF service, like NIFI, into an HDP cluster, the NIFI service won't be upgradable since it's not truly a part of that stack. It won't broadcast its version, and it won't participate in upgrades.