Registered version HDP-2.3.4.0 is not listed

[#1]

I'm going to update HDP-2.2.8.0 to HDP-2.3.4.0 using Ambari.

After I've registered HDP-2.3.4.0 version, it's not displayed on Versions tab in Ambari.

I'm using Ambari 2.2. What am I missing?

[#2]

@Dzianis Frydliand

Maybe the URLs you registered are not accessible. I had the same issue the other day and there was no error, only the version was not registered. Just retry to register the new version again, making sure the repo URLs are correct and accessible.

[#3]

@Predrag Minovic

I've created a local copy of HDP-2.3.4 repo on my network and it's accesible by all nodes.

I did the same when updted 2.2.0 to 2.2.8 and 2.2.9 and it worked fine for me. The issue is only while trying to upgrade 2.2.* to 2.3.*

[#4]

I understand what are you saying, but it's possible that your "Base URLs" don't map exactly your un-tarred directory layout. If you are using the latest HDP-UTILS the layout of directories after untar is a little bit different than before. So, can you please check that. Also check ambari-server log if the registration fails again. Finally, if you are most certain that your Base URLs are correct you can check "Skip repo base URL validation" flag at the bottom of the page, but it's not recommended.

[#5]

By the way, to check your base URLs you can try this, from any node in the cluster:

wget <HDP-base-URL>/repodata/repomd.xml 
wget <HDP-UTILS-base-URL>/repodata/repomd.xml

[#6]

Dear @Predrag Minovic,

Thank you for your answer.

I've made all of the suggested checks, but no result... The version HDP-2.3.4.0 is registered and listed on 'Manage Versions' page (pls, see the attached picture).

When I click on the 'Install on...' button for version HDP-2.3.4.0, I'm transferred back to 'Stack and Versions' page, and I don't see previously registered version HDP-2.3.4.0...

[#7]

Btw, I've switched from local repositories to Hortonworks' ones listed here: http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.0.0/bk_Installing_HDP_AMB/content/_hdp_stack_rep...:

http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.4.0

http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6

[#8]

Hi @Dennis Fridlyand, well at this point I don't have any other idea. Anything in your abmari-server.log? The other day I registered, installed and upgraded from 2.2.8 to 2.3.4, the only difference with your screenshot was that I didn't have 2.2.9 ready to perform upgrade. Though, that page implies that you can register and install multiple versions. Is an upgrade to 2.2.9 first and then to 2.3.4 an option? Re repos: public or local should not make any difference as far as the links are accessible and correct. I always use local because the install is much faster.

[#9]

@Dennis Fridlyand

why don't upgrade to 2.2.9.0 first, the current 2.2.8 will be de-registered and try again then? Also consider trying Ambari 2.2.1.0 that was just released. I'm guessing but I think Ambari can only maintain 4 versions of HDP and you have all four, you might need to drop one before upgrading to 2.3.4. Wierder things have happened...

[#10]

@Artem Ervits

I'll try to use ambari 2.2.1.0

It seems to me that I'm facing https://issues.apache.org/jira/browse/AMBARI-10603 issue

[#11]

well it says it was fixed in 2.1.0 and you're on 2.2.0. Can you confirm all your agents are on 2.2.0.0 as well? @Dennis Fridlyand

[#12]

@Artem Ervits

Here's output from server and datanodes:

[root@h1 data]# rpm -qa | grep ambari
ambari-metrics-hadoop-sink-2.2.0.0-1310.x86_64
ambari-server-2.2.0.0-1310.x86_64
ambari-log4j-1.7.0.169-1.noarch
ambari-agent-2.2.0.0-1310.x86_64
ambari-metrics-monitor-2.2.0.0-1310.x86_64
ambari-metrics-collector-2.2.0.0-1310.x86_64


[root@h4 ~]# rpm -qa | grep ambari
ambari-metrics-hadoop-sink-2.2.0.0-1310.x86_64
ambari-log4j-1.7.0.169-1.noarch
ambari-metrics-monitor-2.2.0.0-1310.x86_64
ambari-agent-2.2.0.0-1310.x86_64

[#13]

Definitely remove 1.7 log4j and run Ambari agent reinstall. It may not be the issue but we don't know unintended consequences. @Dennis Fridlyand

[#14]

@Artem Ervits

Also I'd like to mention the following output from API:

curl -i -k -u <user>:<pass> -X GET http://<server>:8080/api/v1/stacks/HDP/versions/2.2/compatible_repository_versions?fields=*,repo_id,...

[root@h1 data]# curl -i -k -u <user>:<pass> -X GET http://<server>:8080/api/v1/stacks/HDP/versions/2.2/compatible_repository_versions?fields=*,repo_id,...
{
  "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.2/compatible_repository_versions?fields=*,repo_id,repo_name,stack_version",
  "items" : [
    {
      "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.2/compatible_repository_versions/1",
      "CompatibleRepositoryVersions" : {
        "display_name" : "HDP-2.2.0.0-2041",
        "id" : 1,
        "repository_version" : "2.2.0.0-2041",
        "stack_name" : "HDP",
        "stack_version" : "2.2",
        "upgrade_types" : [
          "ROLLING",
          "NON_ROLLING"
        ]
      },
      "operating_systems" : [
        {
          "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.2/compatible_repository_versions/1/operating_systems/redhat6",
          "OperatingSystems" : {
            "os_type" : "redhat6",
            "repository_version_id" : 1,
            "stack_name" : "HDP",
            "stack_version" : "2.2"
          }
        },
        {
          "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.2/compatible_repository_versions/1/operating_systems/suse11",
          "OperatingSystems" : {
            "os_type" : "suse11",
            "repository_version_id" : 1,
            "stack_name" : "HDP",
            "stack_version" : "2.2"
          }
        },
        {
          "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.2/compatible_repository_versions/1/operating_systems/ubuntu12",
          "OperatingSystems" : {
            "os_type" : "ubuntu12",
            "repository_version_id" : 1,
            "stack_name" : "HDP",
            "stack_version" : "2.2"
          }
        }
      ]
    },
    {
      "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.2/compatible_repository_versions/53",
      "CompatibleRepositoryVersions" : {
        "display_name" : "HDP-2.2.8.0",
        "id" : 53,
        "repository_version" : "2.2.8.0-3150",
        "stack_name" : "HDP",
        "stack_version" : "2.2",
        "upgrade_types" : [
          "ROLLING",
          "NON_ROLLING"
        ]
      },
      "operating_systems" : [
        {
          "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.2/compatible_repository_versions/53/operating_systems/redhat6",
          "OperatingSystems" : {
            "os_type" : "redhat6",
            "repository_version_id" : 53,
            "stack_name" : "HDP",
            "stack_version" : "2.2"
          }
        }
      ]
    },
    {
      "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.2/compatible_repository_versions/102",
      "CompatibleRepositoryVersions" : {
        "display_name" : "HDP-2.2.9.0",
        "id" : 102,
        "repository_version" : "2.2.9.0-3393",
        "stack_name" : "HDP",
        "stack_version" : "2.2",
        "upgrade_types" : [
          "ROLLING",
          "NON_ROLLING"
        ]
      },
      "operating_systems" : [
        {
          "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.2/compatible_repository_versions/102/operating_systems/redhat6",
          "OperatingSystems" : {
            "os_type" : "redhat6",
            "repository_version_id" : 102,
            "stack_name" : "HDP",
            "stack_version" : "2.2"
          }
        }
      ]
    }
  ]
}

Please note that version 2.3 are not listed in the output.

When changing 2.2 to 2.3 in the request, I'm getting the following output:

[root@h1 data]# curl -i -k -u <user>:<pass> -X GET http://<server>:8080/api/v1/stacks/HDP/versions/2.3/compatible_repository_versions?fields=*,repo_id,...


{
  "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.3/compatible_repository_versions?fields=*,repo_id,repo_name,stack_version",
  "items" : [
    {
      "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.3/compatible_repository_versions/151",
      "CompatibleRepositoryVersions" : {
        "display_name" : "HDP-2.3.4.0",
        "id" : 151,
        "repository_version" : "2.3.4.0",
        "stack_name" : "HDP",
        "stack_version" : "2.3",
        "upgrade_types" : [
          "ROLLING",
          "NON_ROLLING"
        ]
      },
      "operating_systems" : [
        {
          "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.3/compatible_repository_versions/151/operating_systems/redhat6",
          "OperatingSystems" : {
            "os_type" : "redhat6",
            "repository_version_id" : 151,
            "stack_name" : "HDP",
            "stack_version" : "2.3"
          }
        }
      ]
    },
    {
      "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.3/compatible_repository_versions/201",
      "CompatibleRepositoryVersions" : {
        "display_name" : "HDP-2.3.0.0",
        "id" : 201,
        "repository_version" : "2.3.0.0",
        "stack_name" : "HDP",
        "stack_version" : "2.3",
        "upgrade_types" : [
          "ROLLING",
          "NON_ROLLING"
        ]
      },
      "operating_systems" : [
        {
          "href" : "http://<server>:8080/api/v1/stacks/HDP/versions/2.3/compatible_repository_versions/201/operating_systems/redhat6",
          "OperatingSystems" : {
            "os_type" : "redhat6",
            "repository_version_id" : 201,
            "stack_name" : "HDP",
            "stack_version" : "2.3"
          }
        }
      ]
    }
  ]
}

[#15]

Some nodes are 2.3.4 and some 2.3.0? I'm on mobile phone hard to see. Try to finalize upgrade with hdp set version. I would also suggest contacting support. @Dennis Fridlyand

[#16]

Hey,

I'm facing the same issue:

"When I click on the 'Install on...' button for version HDP-2.3.4.0, I'm transferred back to 'Stack and Versions' page, and I don't see previously registered version HDP-2.3.4.0..."

but with upgrading 2.3.4.0 to 2.4.0.0 using Ambari 2.2.1.1! Any hints on how to resolve this problem?

[#17]

The provided workaround by @Alessio Ubaldi seems to work. But you should also try to first upgrade to 2.3.7.

[#18]

Hi guys,

I had the same problem with ambari 2.2.1.1 and upgrade from 2.3.4.0 to 2.4.0.0 (equal to @Sebastian Zimmermann), I solved it by going to edit the dabatase of Ambari

First, I added the version 2.4.0.0 via Ambari->stack and version, I follow hortonworks guide at http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.0/bk_upgrading_Ambari/content/_upgrading_HDP_r...

Then I analyzed the tables of ambaridb on mysql::

mysql> select * from clusters;
| cluster_id | resource_id | cluster_info | cluster_name | provisioning_state | security_type | desired_cluster_state | desired_stack_id |
|          2 |           4 |              | nb_cluster   | INSTALLED          | NONE          |                       |                4 |
1 row in set (0,00 sec)



mysql> select * from stack;
+----------+------------+-----------------+
| stack_id | stack_name | stack_version   |
+----------+------------+-----------------+
|        8 | HDP        | 2.0             |
|        2 | HDP        | 2.0.6           |
|        1 | HDP        | 2.0.6.GlusterFS |
|        7 | HDP        | 2.1             |
|        3 | HDP        | 2.1.GlusterFS   |
|        5 | HDP        | 2.2             |
|        4 | HDP        | 2.3             |
|        6 | HDP        | 2.3.GlusterFS   |
|        9 | HDP        | 2.4             |
+----------+------------+-----------------+
mysql> select * from repo_version;
| repo_version_id | stack_id | version      | display_name     | repositories                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
|               1 |        4 | 2.3.4.0-3485 | HDP-2.3.4.0-3485 | [{"repositories":[{"Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/updates/2.3.4.0","Repositories/repo_name":"HDP","Repositories/repo_id":"HDP-2.3"},{"Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/suse11sp3","Repositories/repo_name":"HDP-UTILS","Repositories/repo_id":"HDP-UTILS-1.1.0.20"}],"OperatingSystems/os_type":"suse11"},{"repositories":[{"Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.3.4.0","Repositories/repo_name":"HDP","Repositories/repo_id":"HDP-2.3"},{"Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos7","Repositories/repo_name":"HDP-UTILS","Repositories/repo_id":"HDP-UTILS-1.1.0.20"}],"OperatingSystems/os_type":"redhat7"},{"repositories":[{"Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP/debian7/2.x/updates/2.3.4.0","Repositories/repo_name":"HDP","Repositories/repo_id":"HDP-2.3"},{"Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/debian6","Repositories/repo_name":"HDP-UTILS","Repositories/repo_id":"HDP-UTILS-1.1.0.20"}],"OperatingSystems/os_type":"debian7"},{"repositories":[{"Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.4.0","Repositories/repo_name":"HDP","Repositories/repo_id":"HDP-2.3"},{"Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6","Repositories/repo_name":"HDP-UTILS","Repositories/repo_id":"HDP-UTILS-1.1.0.20"}],"OperatingSystems/os_type":"redhat6"},{"repositories":[{"Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.3.4.0","Repositories/repo_name":"HDP","Repositories/repo_id":"HDP-2.3"},{"Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu12","Repositories/repo_name":"HDP-UTILS","Repositories/repo_id":"HDP-UTILS-1.1.0.20"}],"OperatingSystems/os_type":"ubuntu14"},{"repositories":[{"Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x/updates/2.3.4.0","Repositories/repo_name":"HDP","Repositories/repo_id":"HDP-2.3"},{"Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu12","Repositories/repo_name":"HDP-UTILS","Repositories/repo_id":"HDP-UTILS-1.1.0.20"}],"OperatingSystems/os_type":"ubuntu12"}] |
|               2 |        9 | 2.4.0.0      | HDP-2.4.0.0      | [{"repositories":[{"Repositories/repo_id":"HDP-2.4","Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP/debian7/2.x/updates/2.4.0.0/","Repositories/repo_name":"HDP"},{"Repositories/repo_id":"HDP-UTILS-1.1.0.20","Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/debian7","Repositories/repo_name":"HDP-UTILS"}],"OperatingSystems/os_type":"debian7"},{"repositories":[{"Repositories/repo_id":"HDP-2.4","Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.4.0.0","Repositories/repo_name":"HDP"},{"Repositories/repo_id":"HDP-UTILS-1.1.0.20","Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6","Repositories/repo_name":"HDP-UTILS"}],"OperatingSystems/os_type":"redhat6"},{"repositories":[{"Repositories/repo_id":"HDP-2.4","Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.4.0.0","Repositories/repo_name":"HDP"},{"Repositories/repo_id":"HDP-UTILS-1.1.0.20","Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos7","Repositories/repo_name":"HDP-UTILS"}],"OperatingSystems/os_type":"redhat7"},{"repositories":[{"Repositories/repo_id":"HDP-2.4","Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/updates/2.4.0.0/","Repositories/repo_name":"HDP"},{"Repositories/repo_id":"HDP-UTILS-1.1.0.20","Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/suse11sp3","Repositories/repo_name":"HDP-UTILS"}],"OperatingSystems/os_type":"suse11"},{"repositories":[{"Repositories/repo_id":"HDP-2.4","Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x/updates/2.4.0.0/","Repositories/repo_name":"HDP"},{"Repositories/repo_id":"HDP-UTILS-1.1.0.20","Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu12","Repositories/repo_name":"HDP-UTILS"}],"OperatingSystems/os_type":"ubuntu12"},{"repositories":[{"Repositories/repo_id":"HDP-2.4","Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.4.0.0/","Repositories/repo_name":"HDP"},{"Repositories/repo_id":"HDP-UTILS-1.1.0.20","Repositories/base_url":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu14","Repositories/repo_name":"HDP-UTILS"}],"OperatingSystems/os_type":"ubuntu14"}] |
2 rows in set (0,00 sec)


mysql> select * from cluster_version;
+----+-----------------+------------+---------+---------------+---------------+------------+
| id | repo_version_id | cluster_id | state   | start_time    | end_time      | user_name  |
+----+-----------------+------------+---------+---------------+---------------+------------+
|  1 |               1 |          2 | CURRENT | 1452865243126 | 1452865243194 | _anonymous |
+----+-----------------+------------+---------+---------------+---------------+------------+
1 row in set (0,00 sec)

So I added the following information for manually add the stack 2.4 (care to use correct value of repo_version_id and cluster_id):

mysql> insert into cluster_version (id,repo_version_id,cluster_id,state,start_time,end_time,user_name) value (2,2,2,'INSTALLED',1452865243126,1459763540455,'xxxxxx');
Query OK, 1 row affected (0,10 sec)
mysql> select * from cluster_version;
+----+-----------------+------------+-----------+---------------+---------------+------------+
| id | repo_version_id | cluster_id | state     | start_time    | end_time      | user_name  |
+----+-----------------+------------+-----------+---------------+---------------+------------+
|  1 |               1 |          2 | CURRENT   | 1452865243126 | 1452865243194 | _anonymous |
|  2 |               2 |          2 | INSTALLED | 1452865243126 | 1459763540455 | xxxxxx     |
+----+-----------------+------------+-----------+---------------+---------------+------------+
2 rows in set (0,00 sec)

I think we can put the field were in a state NOT INSTALL, but could not find the correct syntax.

When I refresh page appeared to me the stack:

Because the packages is not installed, i think is MUST click Reinstall Packages before click Perform Upgrade.

The Install Packages took place without problems.

After install packages state of host change in installed, but if try to upgrade I can't select rolling or express upgrade and can't click on proceed.

Any idea for resolve the issue?

[#19]

I used your solution, but I rebooted my sandbox vm. After that the wizard worked fine. Tks

[#20]

I upgraded to 2.3.4.7-4 - same behaviour as described :(.

As you can see with @Alessio Ubaldi last screenshot, the workaround is not working properly, as he is unable to start the upgrade process!

[#21]

Hi everyone,

After facing this issue and posting this topic I've decided to start over and made a clean install. I've just had a test cluster and this was the shortets way after all.

But today I've started to upgrade my cluster from HDP-2.3.4.0-3485 то HDP-2.4.0.0.

First, I've updated to version HDP-2.3.4.7 as was suggested in official documentation. Also I've updates Ambari to version 2.2.1.1 (the latest available).

Then I've tried to register HDP-2.4.0.0 version, but faced the described issue AGAIN!!!

Please, help! Has anyone updated from 2.3 to 2.4 stack? Are there any walkarounds that work?