I am testing deploying an HDF cluster with Ambari. For my test I have two nodes, master and ingest-1. Both boxes are running Ubuntu 16.04. I went through the Ambari wizard for creating a new cluster and got to the step labeled Install, Start and Test. The first step in the install is DRPC Server Install where it failed.
stderr gave has the following.
2019-08-26 16:12:49,573 - The 'storm-client' component did not advertise a version. This may indicate a problem with the component packaging.
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stacks/HDF/3.3/services/STORM/package/scripts/drpc_server.py", line 86, in <module>
DrpcServer().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDF/3.3/services/STORM/package/scripts/drpc_server.py", line 39, in install
self.install_packages(env)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 839, in install_packages
name = self.format_package_name(package['name'])
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 562, in format_package_name
return self.get_package_from_available(name)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 529, in get_package_from_available
raise Fail("No package found for {0}(expected name: {1})".format(name, name_with_version))
resource_management.core.exceptions.Fail: No package found for storm-${stack_version}(expected name: storm-3-4-1-1)
The last relevant lines of the logs are as follows.
2019-08-26 16:12:38,484 - Writing File['/etc/apt/sources.list.d/ambari-hdf-3.list'] because it doesn't exist
2019-08-26 16:12:38,484 - call[['apt-get', 'update', '-qq', '-o', u'Dir::Etc::sourcelist=sources.list.d/ambari-hdf-3.list', '-o', 'Dir::Etc::sourceparts=-', '-o', 'APT::Get::List-Cleanup=0']] {'sudo': True, 'quiet': False}
2019-08-26 16:12:38,812 - call returned (0, '')
2019-08-26 16:12:38,813 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2019-08-26 16:12:38,853 - Installing package unzip ('/usr/bin/apt-get -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume-yes install unzip')
2019-08-26 16:12:42,762 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2019-08-26 16:12:42,796 - Skipping installation of existing package curl
2019-08-26 16:12:42,796 - Package['hdf-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2019-08-26 16:12:42,863 - Installing package hdf-select ('/usr/bin/apt-get -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume-yes install hdf-select')
2019-08-26 16:12:45,195 - call[('ambari-python-wrap', u'/usr/bin/hdf-select', 'versions')] {}
2019-08-26 16:12:45,213 - call returned (1, 'Traceback (most recent call last):\n File "/usr/bin/hdf-select", line 406, in <module>\n printVersions()\n File "/usr/bin/hdf-select", line 251, in printVersions\n for f in os.listdir(root):\nOSError: [Errno 2] No such file or directory: \'/usr/hdf\'')
2019-08-26 16:12:45,404 - Command repositories: HDF-3.4-repo-3, HDP-UTILS-1.1.0.22-repo-3
2019-08-26 16:12:45,404 - Applicable repositories: HDF-3.4-repo-3, HDP-UTILS-1.1.0.22-repo-3
2019-08-26 16:12:49,429 - Looking for matching packages in the following repositories: https:__public-repo-1.hortonworks.com_HDF_ubuntu16_3.x_updates_3.4.1.1, https:__public-repo-1.hortonworks.com_HDP-UTILS-1.1.0.22_repos_ubuntu16
2019-08-26 16:12:49,555 - call[('ambari-python-wrap', u'/usr/bin/hdf-select', 'versions')] {}
2019-08-26 16:12:49,572 - call returned (1, 'Traceback (most recent call last):\n File "/usr/bin/hdf-select", line 406, in <module>\n printVersions()\n File "/usr/bin/hdf-select", line 251, in printVersions\n for f in os.listdir(root):\nOSError: [Errno 2] No such file or directory: \'/usr/hdf\'')
2019-08-26 16:12:49,573 - The 'storm-client' component did not advertise a version. This may indicate a problem with the component packaging.
Command failed after 1 tries
As the message indicates the directory /usr/hdf does not exist on ingest-1.
⟫ ls /usr/
bin games include lib local sbin share src
The error can easily be reproduced on the command line for ingest-1.
⟫ hdf-select versions
Traceback (most recent call last):
File "/usr/bin/hdf-select", line 406, in
printVersions()
File "/usr/bin/hdf-select", line 251, in printVersions
for f in os.listdir(root):
OSError: [Errno 2] No such file or directory: '/usr/hdf'
Both of these machines are brand new standard installations of Ubuntu 16.04. I have only installed the packages and made the recommended changes as outlined in the documentation.
