Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to add services thorugh ambari in hortonworks HDP 2.6

Highlighted

Unable to add services thorugh ambari in hortonworks HDP 2.6

Explorer

i have setup multinode cluster with hortonworks HDP 2.6 version. I am trying to add flume service on my cluster and i am getting following error,

stderr: /var/lib/ambari-agent/data/errors-5518.txt

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py", line 122, in <module>
    FlumeHandler().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 375, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py", line 45, in install
    self.install_packages(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 821, in install_packages
    retry_count=agent_stack_retry_count)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 53, in action_install
    self.install_package(package_name, self.resource.use_repos, self.resource.skip_repos)
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/apt.py", line 75, in wrapper
    return function_to_decorate(self, name, *args[2:])
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/apt.py", line 376, in install_package
    self.checked_call_with_retries(cmd, sudo=True, env=INSTALL_CMD_ENV, logoutput=self.get_logoutput())
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 266, in checked_call_with_retries
    return self._call_with_retries(cmd, is_checked=True, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 283, in _call_with_retries
    code, out = func(cmd, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bin/apt-get -q -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume-yes install flume-2-6-4-0-91' returned 100. Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  flume-2-6-4-0-91
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 74.0 MB of archives.
After this operation, 86.6 MB of additional disk space will be used.
Get:1 http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0 HDP/main amd64 flume-2-6-4-0-91 all 1.5.2.2.6.4.0-91 [74.0 MB]
Err:1 http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0 HDP/main i386 flume-2-6-4-0-91 all 1.5.2.2.6.4.0-91
  Hash Sum mismatch
Get:1 http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0 HDP/main i386 flume-2-6-4-0-91 all 1.5.2.2.6.4.0-91 [74.0 MB]
Err:1 http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0 HDP/main i386 flume-2-6-4-0-91 all 1.5.2.2.6.4.0-91
  Hash Sum mismatch
Fetched 400 B in 0s (746 B/s)
E: Failed to fetch http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0/pool/main/f/flume/flume-2-6-4-...  Hash Sum mismatch

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

stdout: /var/lib/ambari-agent/data/output-5518.txt

2018-05-02 13:25:19,415 - Stack Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command Version=None -> 2.6
2018-05-02 13:25:19,419 - Using hadoop conf dir: /usr/hdp/2.6.4.0-91/hadoop/conf
2018-05-02 13:25:19,419 - Group['hdfs'] {}
2018-05-02 13:25:19,420 - Group['hadoop'] {}
2018-05-02 13:25:19,420 - Group['users'] {}
2018-05-02 13:25:19,420 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-05-02 13:25:19,421 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-05-02 13:25:19,421 - User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-05-02 13:25:19,422 - User['oozie'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users'], 'uid': None}
2018-05-02 13:25:19,422 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-05-02 13:25:19,423 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users'], 'uid': None}
2018-05-02 13:25:19,423 - User['accumulo'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-05-02 13:25:19,424 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users'], 'uid': None}
2018-05-02 13:25:19,424 - User['flume'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-05-02 13:25:19,425 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hdfs'], 'uid': None}
2018-05-02 13:25:19,425 - User['sqoop'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-05-02 13:25:19,426 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-05-02 13:25:19,426 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-05-02 13:25:19,427 - User['hbase'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-05-02 13:25:19,427 - User['hcat'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-05-02 13:25:19,428 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2018-05-02 13:25:19,428 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2018-05-02 13:25:19,432 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] due to not_if
2018-05-02 13:25:19,432 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents': True, 'mode': 0775, 'cd_access': 'a'}
2018-05-02 13:25:19,433 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2018-05-02 13:25:19,434 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2018-05-02 13:25:19,434 - call['/var/lib/ambari-agent/tmp/changeUid.sh hbase'] {}
2018-05-02 13:25:19,439 - call returned (0, '1015')
2018-05-02 13:25:19,439 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase 1015'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
2018-05-02 13:25:19,443 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase 1015'] due to not_if
2018-05-02 13:25:19,443 - Group['hdfs'] {}
2018-05-02 13:25:19,443 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hdfs', u'hdfs']}
2018-05-02 13:25:19,444 - FS Type: 
2018-05-02 13:25:19,444 - Directory['/etc/hadoop'] {'mode': 0755}
2018-05-02 13:25:19,452 - File['/usr/hdp/2.6.4.0-91/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2018-05-02 13:25:19,453 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2018-05-02 13:25:19,465 - Repository['HDP-2.6-repo-1'] {'append_to_file': False, 'base_url': 'http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0', 'action': ['create'], 'components': [u'HDP', 'main'], 'repo_template': '{{package_type}} {{base_url}} {{components}}', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None}
2018-05-02 13:25:19,469 - File['/tmp/tmpB_grzd'] {'content': 'deb http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0 HDP main'}
2018-05-02 13:25:19,470 - Writing File['/tmp/tmpB_grzd'] because contents don't match
2018-05-02 13:25:19,470 - File['/tmp/tmpsicpFn'] {'content': StaticFile('/etc/apt/sources.list.d/ambari-hdp-1.list')}
2018-05-02 13:25:19,470 - Writing File['/tmp/tmpsicpFn'] because contents don't match
2018-05-02 13:25:19,470 - File['/etc/apt/sources.list.d/ambari-hdp-1.list'] {'content': StaticFile('/tmp/tmpB_grzd')}
2018-05-02 13:25:19,471 - Writing File['/etc/apt/sources.list.d/ambari-hdp-1.list'] because contents don't match
2018-05-02 13:25:19,471 - checked_call[['apt-get', 'update', '-qq', '-o', u'Dir::Etc::sourcelist=sources.list.d/ambari-hdp-1.list', '-o', 'Dir::Etc::sourceparts=-', '-o', 'APT::Get::List-Cleanup=0']] {'sudo': True, 'quiet': False}
2018-05-02 13:25:20,248 - checked_call returned (0, '')
2018-05-02 13:25:20,249 - Repository['HDP-2.6-GPL-repo-1'] {'append_to_file': True, 'base_url': 'http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu16/2.x/updates/2.6.4.0', 'action': ['create'], 'components': [u'HDP-GPL', 'main'], 'repo_template': '{{package_type}} {{base_url}} {{components}}', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None}
2018-05-02 13:25:20,250 - File['/tmp/tmpAMTk7U'] {'content': 'deb http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0 HDP main\ndeb http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu16/2.x/updates/2.6.4.0 HDP-GPL main'}
2018-05-02 13:25:20,250 - Writing File['/tmp/tmpAMTk7U'] because contents don't match
2018-05-02 13:25:20,250 - File['/tmp/tmpkRe0G7'] {'content': StaticFile('/etc/apt/sources.list.d/ambari-hdp-1.list')}
2018-05-02 13:25:20,250 - Writing File['/tmp/tmpkRe0G7'] because contents don't match
2018-05-02 13:25:20,251 - File['/etc/apt/sources.list.d/ambari-hdp-1.list'] {'content': StaticFile('/tmp/tmpAMTk7U')}
2018-05-02 13:25:20,251 - Writing File['/etc/apt/sources.list.d/ambari-hdp-1.list'] because contents don't match
2018-05-02 13:25:20,251 - checked_call[['apt-get', 'update', '-qq', '-o', u'Dir::Etc::sourcelist=sources.list.d/ambari-hdp-1.list', '-o', 'Dir::Etc::sourceparts=-', '-o', 'APT::Get::List-Cleanup=0']] {'sudo': True, 'quiet': False}
2018-05-02 13:25:20,805 - checked_call returned (0, '')
2018-05-02 13:25:20,805 - Repository['HDP-UTILS-1.1.0.22-repo-1'] {'append_to_file': True, 'base_url': 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu16', 'action': ['create'], 'components': [u'HDP-UTILS', 'main'], 'repo_template': '{{package_type}} {{base_url}} {{components}}', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None}
2018-05-02 13:25:20,807 - File['/tmp/tmpbhTasl'] {'content': 'deb http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0 HDP main\ndeb http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu16/2.x/updates/2.6.4.0 HDP-GPL main\ndeb http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu16 HDP-UTILS main'}
2018-05-02 13:25:20,807 - Writing File['/tmp/tmpbhTasl'] because contents don't match
2018-05-02 13:25:20,807 - File['/tmp/tmptkSbVn'] {'content': StaticFile('/etc/apt/sources.list.d/ambari-hdp-1.list')}
2018-05-02 13:25:20,807 - Writing File['/tmp/tmptkSbVn'] because contents don't match
2018-05-02 13:25:20,807 - File['/etc/apt/sources.list.d/ambari-hdp-1.list'] {'content': StaticFile('/tmp/tmpbhTasl')}
2018-05-02 13:25:20,808 - Writing File['/etc/apt/sources.list.d/ambari-hdp-1.list'] because contents don't match
2018-05-02 13:25:20,808 - checked_call[['apt-get', 'update', '-qq', '-o', u'Dir::Etc::sourcelist=sources.list.d/ambari-hdp-1.list', '-o', 'Dir::Etc::sourceparts=-', '-o', 'APT::Get::List-Cleanup=0']] {'sudo': True, 'quiet': False}
2018-05-02 13:25:21,520 - checked_call returned (0, '')
2018-05-02 13:25:21,520 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-05-02 13:25:21,551 - Skipping installation of existing package unzip
2018-05-02 13:25:21,551 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-05-02 13:25:21,580 - Skipping installation of existing package curl
2018-05-02 13:25:21,581 - Package['hdp-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-05-02 13:25:21,610 - Skipping installation of existing package hdp-select
2018-05-02 13:25:21,613 - The repository with version 2.6.4.0-91 for this command has been marked as resolved. It will be used to report the version of the component which was installed
2018-05-02 13:25:21,823 - Command repositories: HDP-2.6-repo-1, HDP-2.6-GPL-repo-1, HDP-UTILS-1.1.0.22-repo-1
2018-05-02 13:25:21,823 - Applicable repositories: HDP-2.6-repo-1, HDP-2.6-GPL-repo-1, HDP-UTILS-1.1.0.22-repo-1
2018-05-02 13:25:24,496 - Looking for matching packages in the following repositories: public-repo-1.hortonworks.com_HDP_ubuntu16_2.x_updates_2.6.4.0, public-repo-1.hortonworks.com_HDP-GPL_ubuntu16_2.x_updates_2.6.4.0, public-repo-1.hortonworks.com_HDP-UTILS-1.1.0.22_repos_ubuntu16
2018-05-02 13:25:24,497 - Package['flume-2-6-4-0-91'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-05-02 13:25:24,529 - Installing package flume-2-6-4-0-91 ('/usr/bin/apt-get -q -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume-yes install flume-2-6-4-0-91')
2018-05-02 13:25:25,286 - Execution of '/usr/bin/apt-get -q -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume-yes install flume-2-6-4-0-91' returned 100. Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  flume-2-6-4-0-91
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 74.0 MB of archives.
After this operation, 86.6 MB of additional disk space will be used.
Get:1 http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0 HDP/main amd64 flume-2-6-4-0-91 all 1.5.2.2.6.4.0-91 [74.0 MB]
Err:1 http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0 HDP/main i386 flume-2-6-4-0-91 all 1.5.2.2.6.4.0-91
  Hash Sum mismatch
Get:1 http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0 HDP/main i386 flume-2-6-4-0-91 all 1.5.2.2.6.4.0-91 [74.0 MB]
Err:1 http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0 HDP/main i386 flume-2-6-4-0-91 all 1.5.2.2.6.4.0-91
  Hash Sum mismatch
Fetched 400 B in 0s (1298 B/s)
E: Failed to fetch http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.4.0/pool/main/f/flume/flume-2-6-4-...  Hash Sum mismatch

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
2018-05-02 13:25:25,286 - Failed to install package flume-2-6-4-0-91. Executing '/usr/bin/apt-get update -qq'
2018-05-02 13:25:36,417 - Retrying to install package flume-2-6-4-0-91 after 30 seconds
2018-05-02 13:26:07,445 - The repository with version 2.6.4.0-91 for this command has been marked as resolved. It will be used to report the version of the component which was installed

Command failed after 1 tries

Can anyone help me to resolve this?

Thanks in advance

2 REPLIES 2

Re: Unable to add services thorugh ambari in hortonworks HDP 2.6

Super Mentor

@manikandan ayyasamy

Looks like the apt repo is corrupted as we see Hash error

Hash Sum mismatch


Can you please try cleaning up the "apt" content once and then try again:
Example:

# rm -rf /var/lib/apt/lists/* 
# apt-get update
# apt-get clean


(OR) if you do not want to delete the contents then Please take a Backup of the mentioned directory instead of deleting it.

Re: Unable to add services thorugh ambari in hortonworks HDP 2.6

Explorer

I tried your steps again i am getting the same error.My cluster is multi distributed cluster.

Don't have an account?
Coming from Hortonworks? Activate your account here