HDF3 installation problem

[#1]

See my answer about fixing stack_root. "I tried upgrading the management pack after, but didn't help." It looks like you HAVE to do mpack first then Ambari upgrade going from Ambari 2.5.x to Ambari 2.6.x. This will automatically update the stack_root.

[#2]

The issue seems to be Ambari doesn't like the JSON coming back from Stack.get_stack_root(). The format looks like it changed between Ambari 2.5.1 and Ambari 2.5.2 with AMBARI-21430. The commit looks like this: f33a250c0

This is what the new Ambari 2.6 cluster has for stack_root.

"stack_root" : "{\"HDF\":\"/usr/hdf\"}", 

This is what our upgraded Ambari 2.6 cluster has for stack_root.

"stack_root" : "/usr/hdf", 

This can easily be updated with configs.py since this is just cluster-env -> stack_root. An example of doing this:

/var/lib/ambari-server/resources/scripts/configs.py -l $(hostname -f) -t 8081 -s https -n CLUSTER_NAME -u USERNAME -p PASSWORD -a set -c cluster-env -k stack_root -v '{"HDF":"/usr/hdf"}' 

PS: It also looks like this requires that you have the 3.0.2 mpack installed (even if you have 3.0.1 HDF installed) since that is compatible with Ambari 2.6.x (see matrix here). Without the 3.0.2 mpack, we were getting different errors (like the ones in the original question).

To install 3.0.2 mpack (make sure purge flag is set):

sudo ambari-server install-mpack --mpack=https://s3.amazonaws.com/public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.0.2.0/tars/hdf_ambari_mp/hdf-ambari-mpack-3.0.2.0-76.tar.gz --purge --verbose --force

[#3]

Hello @Kevin Risden,

I have not been able to solve it with configs.py, (probably) because I already tried without success other solutions (that may have interfered).

Anyway your hint about inverting the order of the upgrades has been helpful, the correct order is: first upgrade the mpack then upgrade Ambari. Thanks.

On this matter the Hortonworks HDF Upgrade documentation is confusing, it seems to suggest the wrong order (and leaves the idea that you can follow both orders).

Raffaele

[#4]

@Raffaele S - We found a few more things that were broken which we were able to fix. I am working on writing up some more steps for this. We were able to get everything to work. I agree with you that the documentation isn't clear that mpack must go before Ambari upgrade. I'll update this question with more info hopefully soon.

[#5]

@Raffaele S Here are all the details in a blog: https://risdenk.github.io/2018/02/07/hdf-ambari-mpack-upgrades.html

[#6]

Hi @Kevin Risden, I also ran upgrade before updating mpack. Can you please share the information how to fix it?

,

Hi @Kevin Risden, I also ran upgrade before updating mpack. Can you please share the information how to fix it?

[#7]

Did you backup the database, resource folders etc. as described in the upgrade guide?

If you did, restore all the backups, downgrade ambari to the previous version and follow the proper order (as I described above).

[#8]

Unfortunately I haven't done backup prior to upgrade... any suggestion to fix this issue will be really appreciated.

[#9]

@Devendra Singh Here are all the details in a blog: https://risdenk.github.io/2018/02/07/hdf-ambari-mpack-upgrades.html

[#10]

Unfortunately I haven't done backup prior to upgrade... any suggestion to fix this issue will be really appreciated.