Hi @Abdelkrim Hadjidj, Thanks for the quick reply! Good to know that there will be no more HDP 2.3 releases. I had seen the following paragraph in that blog post: Coordinated maintenance releases will be made available to ensure that the common core is maintained. For example, HDP 2.3.4 and HDP 2.4.0 are based on the same common core and when defects need to be addressed on that common core future releases of HDP (such as 2.3.5 and 2.4.1) will be coordinated to maintain synchronicity. so I thought there might be further 2.3 releases. Thanks for clearing it up.
... View more
The HDP 2.4 release post (http://hortonworks.com/blog/announcing-the-general-availability-of-hortonworks-data-platform-2-4/) said that maintenance releases for HDP 2.3 and HDP 2.4 would continue happen in parallel to "maintain synchronicity". However, if I visit the HDP documentation / release page (http://docs.hortonworks.com/index.html), I see that a new maintenance release (2.4.2) of HDP 2.4 has been released, but there is no corresponding 2.3 release (I'm guessing it would be 2.3.6). Are there plans for future 2.3 releases?
... View more
We recently ran into a reverse scan bug in HBase (similar to HBASE-13830 and HBASE-14283). According to HBASE-14283, it has been fixed in version 1.1.3 of HBase, but HDP only supports up to version 1.1.2. The bug is also not on the extra patch list in the HDP documentation (http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_HDP_RelNotes/content/patch_hive.html). Looking at the source code in the public-repo, I noticed that the newest HBase version from Hortonworks (18.104.22.168.4.0.0-169) now includes the fix we need. I tried browsing the HDP public repo using s3cmd (http://henning.kropponline.de/2016/01/24/s3cmd/) to find a release that contained HBase 22.214.171.124.4.0.0-169. I did find a new 126.96.36.199 release folder, but it was missing the repo file and the built rpms. Neither 188.8.131.52 or 184.108.40.206 contained new enough versions of HBase. s3cmd ls s3://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/
s3cmd ls s3://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/220.127.116.11/
2016-03-11 17:45 45 s3://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/18.104.22.168/HDP-22.214.171.124-centos7-deb.tar.gz
2016-03-11 17:45 45 s3://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/126.96.36.199/HDP-188.8.131.52-centos7-tars-tarball.tar.gz
2016-03-11 17:45 836 s3://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/184.108.40.206/HDP-220.127.116.11-centos7-tars-tarball.tar.gz.asc
2016-03-11 17:45 91 s3://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/18.104.22.168/HDP-22.214.171.124-centos7-tars-tarball.tar.gz.md5
Is there any way to manually upgrade the HBase version on our cluster (managed with Ambari 2.2.1), or a way to create the HDP repo by hand?
... View more
I am trying to setup a mirror of the HDP repositories within our local network to reduce our external bandwidth usage and improve cluster installation time. We have many developers deploying HDP locally using ambari-docker, and by default every deployment needs to download several GBs to install HDFS/Yarn/Spark etc.
I've followed the instructions for getting a local mirror of the repository working (http://docs.hortonworks.com/HDPDocuments/Ambari-126.96.36.199/bk_Installing_HDP_AMB/content/_setting_up_a_local_repository.html), and can confirm that everything works. However, these instructions are tied to a specific version of HDP, HDP-Utils, and Ambari. How could I extend those instructions to support all versions? When we deploy the cluster with Ambari, it will by default use the newest build of HDP, which is not in our local mirror. I would like to configure our mirror to periodically pull all versions from the hortonworks servers, so that it can always be used as a local mirror during installation, but I don't know of anywhere to find a list of all available versions.
I have put my code for building the local mirror into a github/docker project if you're curious.
... View more