invalid base url while registering version in Ambari

[#1]

hi there,

I'd like to upgrade HDP2.6.3 to HDP2.6.5 without internet access in the cluster (OS: ubuntu). Local repot files are downloaded,extraced and accessiable by the cluster.

In the last step, The base url are :

HDP-2.6 : http://10.2.26.11/hdp2.6.5/HDP/ubuntu16/2.6.5.0-292

HDP-2.6-GPL : http://10.2.26.11/hdp2.6.5/HDP-GPL/ubuntu16/2.6.5.0-292/hdp-gpl.gpl.list

HDP-UTILS-1.1.0.22: http://10.2.26.11/hdp2.6.5/HDP-UTILS/ubuntu16/1.1.0.22

"HDP-2.6" is successfully, but "HDP-2.6-GPL" and "HDP-UTILS-1.1.0.22" are encoutering error: "Some of the repositories failed validation. Make changes to the base url or skip validation..."

Any advice?

Thanks

[#2]

Hi @forest lin

It looks like server-side validation of This URL's is failing.

can you try to perform the operation with tailf of ambari-server.log you will know whats going wrong.

[root@asnaik1 ~]# tail -f /var/log/ambari-server/ambari-server.log		

hope this helps.

[#3]

26 Jul 2018 19:00:38,853 ERROR [ambari-client-thread-52774] BaseManagementHandler:61 - Caught a system exception while attempting to create a resource: An internal system exception occurred: Stack data, stackName=HDP, stackVersion= 2.6, osType=ubuntu16, repoId= HDP-UTILS-1.1.0.22
org.apache.ambari.server.controller.spi.SystemException: An internal system exception occurred: Stack data, stackName=HDP, stackVersion= 2.6, osType=ubuntu16, repoId= HDP-UTILS-1.1.0.22
        at org.apache.ambari.server.controller.internal.AbstractResourceProvider.createResources(AbstractResourceProvider.java:287)
        at org.apache.ambari.server.controller.internal.RepositoryResourceProvider.createResources(RepositoryResourceProvider.java:217)
        at org.apache.ambari.server.controller.internal.ClusterControllerImpl.createResources(ClusterControllerImpl.java:298)
        at org.apache.ambari.server.api.services.persistence.PersistenceManagerImpl.create(PersistenceManagerImpl.java:97)
        at org.apache.ambari.server.api.handlers.CreateHandler.persist(CreateHandler.java:37)
        at org.apache.ambari.server.api.handlers.BaseManagementHandler.handleRequest(BaseManagementHandler.java:73)
        at org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:144)
        at org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:126)
        at org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:90)
        at org.apache.ambari.server.api.services.RepositoryService.createRepository(RepositoryService.java:100)
        at sun.reflect.GeneratedMethodAccessor763.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)



Caused by: org.apache.ambari.server.StackAccessException: Stack data, stackName=HDP, stackVersion= 2.6, osType=ubuntu16, repoId= HDP-UTILS-1.1.0.22
        at org.apache.ambari.server.api.services.AmbariMetaInfo.getRepository(AmbariMetaInfo.java:411)
        at org.apache.ambari.server.controller.AmbariManagementControllerImpl.verifyRepository(AmbariManagementControllerImpl.java:4621)
        at org.apache.ambari.server.controller.AmbariManagementControllerImpl.verifyRepositories(AmbariManagementControllerImpl.java:4607)
        at org.apache.ambari.server.controller.internal.RepositoryResourceProvider$6.invoke(RepositoryResourceProvider.java:221)
        at org.apache.ambari.server.controller.internal.RepositoryResourceProvider$6.invoke(RepositoryResourceProvider.java:217)
        at org.apache.ambari.server.controller.internal.AbstractResourceProvider.invokeWithRetry(AbstractResourceProvider.java:455)
        at org.apache.ambari.server.controller.internal.AbstractResourceProvider.createResources(AbstractResourceProvider.java:278)
        ... 95 more

26 Jul 2018 11:42:00,631 ERROR [ambari-client-thread-50569] BaseManagementHandler:61 - Caught a system exception while attempting to create a resource: An internal system exception occurred: Stack data, stackName=HDP, stackVersion= 2.6, osType=ubuntu16, repoId= HDP-2.6-GPL
org.apache.ambari.server.controller.spi.SystemException: An internal system exception occurred: Stack data, stackName=HDP, stackVersion= 2.6, osType=ubuntu16, repoId= HDP-2.6-GPL
        at org.apache.ambari.server.controller.internal.AbstractResourceProvider.createResources(AbstractResourceProvider.java:287)
        at org.apache.ambari.server.controller.internal.RepositoryResourceProvider.createResources(RepositoryResourceProvider.java:217)
        at org.apache.ambari.server.controller.internal.ClusterControllerImpl.createResources(ClusterControllerImpl.java:298)
        at org.apache.ambari.server.api.services.persistence.PersistenceManagerImpl.create(PersistenceManagerImpl.java:97)
        at org.apache.ambari.server.api.handlers.CreateHandler.persist(CreateHandler.java:37)
        at org.apache.ambari.server.api.handlers.BaseManagementHandler.handleRequest(BaseManagementHandler.java:73)
        at org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:144)
        at org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:126)
        at org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:90)
        at org.apache.ambari.server.api.services.RepositoryService.createRepository(RepositoryService.java:100)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java



Caused by: org.apache.ambari.server.StackAccessException: Stack data, stackName=HDP, stackVersion= 2.6, osType=ubuntu16, repoId= HDP-2.6-GPL
        at org.apache.ambari.server.api.services.AmbariMetaInfo.getRepository(AmbariMetaInfo.java:411)
        at org.apache.ambari.server.controller.AmbariManagementControllerImpl.verifyRepository(AmbariManagementControllerImpl.java:4621)
        at org.apache.ambari.server.controller.AmbariManagementControllerImpl.verifyRepositories(AmbariManagementControllerImpl.java:4607)
        at org.apache.ambari.server.controller.internal.RepositoryResourceProvider$6.invoke(RepositoryResourceProvider.java:221)
        at org.apache.ambari.server.controller.internal.RepositoryResourceProvider$6.invoke(RepositoryResourceProvider.java:217)
        at org.apache.ambari.server.controller.internal.AbstractResourceProvider.invokeWithRetry(AbstractResourceProvider.java:455)
        at org.apache.ambari.server.controller.internal.AbstractResourceProvider.createResources(AbstractResourceProvider.java:278)
        ... 96 more

[#4]

Hi @forest lin ,

By the exception trace, I understand you are installing the HDP-2.6.5 in Ambari-2.5.x versions.

Please correct if my understanding is not correct.

Ambari-2.5.x versions are not supposed to work with HDP-2.6.4 + versions. but it can be working with following workaround.

you need to remove the GPL-repo tag from VDF File , place the Hadoop lzo packages(GPL RELATED) inside of the cloned HDP repo only After placing the RPMs in the repo, the repo should be rebuilt so that the metadata includes them and yum can find everything.

Also, rename the HDP-UTILS-1.1.0.22 to HDP-UTILS-1.1.0.21.

i have once faced the same issue and tweaked in this way.

Hope this works.

If no if you can upgrade to Ambari-2.6.2 that will be best solution to upgrade to HDP-2.6.5

[#5]

my Ambari version is 2.6.0.0.

The <tag> is commetted in the VDF file HDP-2.6.5.0-292.xml to fix the "invalid tag" issue while uploading the VDF in the 1st step.. Is it causing error afterwards?hdp-2650-292.xml

[#6 · Accepted solution]

It's caused by missing file in folder dists in HDP-UTILS-1.1.0.22-ubuntu16.tar.gz and fixed as following after getting the advice from a HDP consultant

1. extract HDP-UTILS-1.1.0.22-ubuntu16.tar.gz

2. mkdir HDP-UTILS under folder dists

3. copy all files from HDP repot /hdp/HDP/ubuntu16/2.6.5.0-292/dists/HDP/ to above HDP-UTILS folder

4 update the HDP-UTILS repo base url according to https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.0.0/bk_ambari-installation/content/setting_up_a...