Could not start services after successfully upgradation HDP 2.6.4 to 2.6.5

[#1]

Hi Folks,

Yesterday evening i had successfully done up-gradation HDP 2.6.4 to 2.6.5 but today when i started all services:

i found following message in logs:

2018-12-12 06:01:03,067 - File['/etc/yum.repos.d/ambari-hdp-51.repo'] {'content': '[HDP-2.6-repo-51]\nname=HDP-2.6-repo-51\nbaseurl=http://10.26.35.16/iso/2.6.5/HDP/centos7/2.6.5.0-292\n\npath=/\nenabled=1\ngpgcheck=0'}
2018-12-12 06:01:03,068 - Writing File['/etc/yum.repos.d/ambari-hdp-51.repo'] because contents don't match
2018-12-12 06:01:03,068 - Repository with url  is not created due to its tags: set([u'GPL'])
2018-12-12 06:01:03,069 - Repository['HDP-UTILS-1.1.0.22-repo-51'] {'append_to_file': True, 'base_url': 'http://10.26.35.16/iso/2.6.5/HDP-UTILS/centos7/1.1.0.22', 'action': ['create'], 'components': [u'HDP-UTILS', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-51', 'mirror_list': None}
2018-12-12 06:01:03,074 - File['/etc/yum.repos.d/ambari-hdp-51.repo'] {'content': '[HDP-2.6-repo-51]\nname=HDP-2.6-repo-51\nbaseurl=http://10.26.35.16/iso/2.6.5/HDP/centos7/2.6.5.0-292\n\npath=/\nenabled=1\ngpgcheck=0\n[HDP-UTILS-1.1.0.22-repo-51]\nname=HDP-UTILS-1.1.0.22-repo-51\nbaseurl=http://10.26.35.16/iso/2.6.5/HDP-UTILS/centos7/1.1.0.22\n\npath=/\nenabled=1\ngpgcheck=0'}
2018-12-12 06:01:03,074 - Writing File['/etc/yum.repos.d/ambari-hdp-51.repo'] because contents don't match
2018-12-12 06:01:03,075 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-12-12 06:01:03,181 - Skipping installation of existing package unzip
2018-12-12 06:01:03,181 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-12-12 06:01:03,192 - Skipping installation of existing package curl
2018-12-12 06:01:03,192 - Package['hdp-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-12-12 06:01:03,208 - Skipping installation of existing package hdp-select
2018-12-12 06:01:03,217 - The repository with version 2.6.5.0-292 for this command has been marked as resolved. It will be used to report the version of the component which was installed
2018-12-12 06:01:03,533 - Stack Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command Version=None -> 2.6
2018-12-12 06:01:03,543 - Using hadoop conf dir: /usr/hdp/2.6.5.0-292/hadoop/conf
2018-12-12 06:01:03,548 - checked_call['hostid'] {}
2018-12-12 06:01:03,552 - checked_call returned (0, '1a0a1023')
2018-12-12 06:01:03,561 - Command repositories: HDP-2.6-repo-51, HDP-2.6-GPL-repo-51, HDP-UTILS-1.1.0.22-repo-51
2018-12-12 06:01:03,562 - Applicable repositories: HDP-2.6-repo-51, HDP-2.6-GPL-repo-51, HDP-UTILS-1.1.0.22-repo-51
2018-12-12 06:01:03,565 - Looking for matching packages in the following repositories: HDP-2.6-repo-51, HDP-2.6-GPL-repo-51, HDP-UTILS-1.1.0.22-repo-51
2018-12-12 06:01:06,299 - No package found for hbase_${stack_version}(hbase_(\d|_)+$)
2018-12-12 06:01:06,303 - The repository with version 2.6.5.0-292 for this command has been marked as resolved. It will be used to report the version of the component which was installed

Command failed after 1 tries


Same error occurred for multiple service.
Could you please suggest if i had done any wrong upgrdation?


Thanks
Vinay K

[#2 · Accepted solution]

@Vinay

Can you disable the HDP* repos by changing the path enable from 1 to zero see below

[HDP-2.6-repo-51] 
name=HDP-2.6-repo-51 
baseurl=http://10.26.35.16/iso/2.6.5/HDP/centos7/2.6.5.0-292 
path=enabled=0 
gpgcheck=0'}

Then run

# yum clean all

And try restarting the services through Ambari ,

[#3]

Thanks @Geoffrey Shelton Okot

Also i have found the problem.

Whenever we install new HDP or upgrade HDP, we specify the repository path of HDP and HDP-UTILS in UI, accordingly ambari create repo on all agents with name of HDP-2.6-repo-51, HDP-UTILS-1.1.0.22-repo-51.

But i had also created HDP and HDP-UTILS repository manually on all nodes and all HDP packages had installed with manually repository path. When i was starting services, hbase client and other client find HDP-2.6-repo-51 repository for install the client which i was not getting. Now i have disable the manually repository and reinstall the client package manually.

It's working fine.