Member since
07-14-2017
5
Posts
0
Kudos Received
0
Solutions
07-17-2017
09:03 PM
I was able to resolve the proxy issues, however now I seem to be having versioning issue. Looking at the output log, its trying to install hadoop_2_3_2_0_2950, whereas i have HDP 2.6.1. I suspect hdp-select to be the issue. What are your thoughts?
... View more
07-17-2017
01:30 PM
@Jay SenSharma thanks for the prompt reply. I've tried to wget, and here's the content of the file is: Invalid Arch I am on the other hand able to wget google without any issues. I've also checked the yum.config file and the proxy details are there [main]
cachedir=/var/cache/yum/$basearch/$releasever
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
gpgcheck=1
plugins=1
installonly_limit=5
bugtracker_url=http://bugs.centos.org/set_project.php?project_id=16&ref=http://b$
distroverpkg=centos-release
proxy=http://<ip>:<port>
proxy_username=<username>
proxy_password=<password>
... View more
07-14-2017
09:30 PM
Hi all, I've set up a local repository using httpd, that hosts Amabari 2.5.1, HDP 2.6.1 and HDP_UTILS 1.1.0.21. I have 5 nodes using CentOS 6.5/RHEL 6, and the installation fails during "Install, Start and Test" step. Below is an output from one of the hosts installing DataNode. stderr: /var/lib/ambari-agent/data/errors-94.txt 2017-07-14 16:47:10,887 - Could not find stack selector for stack: HDP
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py", line 161, in <module>
DataNode().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py", line 55, in install
self.install_packages(env)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 693, in install_packages
retry_count=agent_stack_retry_count)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 54, in action_install
self.install_package(package_name, self.resource.use_repos, self.resource.skip_repos)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 51, in install_package
self.checked_call_with_retries(cmd, sudo=True, logoutput=self.get_logoutput())
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 86, in checked_call_with_retries
return self._call_with_retries(cmd, is_checked=True, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 98, in _call_with_retries
code, out = func(cmd, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 -y install hadoop_2_3_2_0_2950' returned 1. Error: Cannot find a valid baseurl for repo: base
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os error was
14: PYCURL ERROR 7 - "Failed to connect to 2001:4178:5:200::10: Network is unreachable" stdout: /var/lib/ambari-agent/data/output-94.txt
2017-07-14 16:46:09,584 - Stack Feature Version Info: stack_version=2.6, version=None, current_cluster_version=None -> 2.6
2017-07-14 16:46:09,586 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
User Group mapping (user_group) is missing in the hostLevelParams
2017-07-14 16:46:09,587 - Group['hadoop'] {}
2017-07-14 16:46:09,588 - Group['users'] {}
2017-07-14 16:46:09,589 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-14 16:46:09,589 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-14 16:46:09,590 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']}
2017-07-14 16:46:09,590 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-14 16:46:09,591 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-14 16:46:09,592 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-14 16:46:09,592 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2017-07-14 16:46:09,594 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2017-07-14 16:46:09,600 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2017-07-14 16:46:09,600 - Group['hdfs'] {}
2017-07-14 16:46:09,601 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'hdfs']}
2017-07-14 16:46:09,601 - FS Type:
2017-07-14 16:46:09,601 - Directory['/etc/hadoop'] {'mode': 0755}
2017-07-14 16:46:09,602 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2017-07-14 16:46:09,627 - Initializing 2 repositories
2017-07-14 16:46:09,627 - Repository['HDP-2.6'] {'base_url': 'http://10.161.0.15/HDP/centos6/', 'action': ['create'], 'components': ['HDP', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP', 'mirror_list': None}
2017-07-14 16:46:09,637 - File['/etc/yum.repos.d/HDP.repo'] {'content': '[HDP-2.6]\nname=HDP-2.6\nbaseurl=http://10.161.0.15/HDP/centos6/\n\npath=/\nenabled=1\ngpgcheck=0'}
2017-07-14 16:46:09,638 - Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://10.161.0.15/HDP-UTILS/', 'action': ['create'], 'components': ['HDP-UTILS', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS', 'mirror_list': None}
2017-07-14 16:46:09,643 - File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content': '[HDP-UTILS-1.1.0.21]\nname=HDP-UTILS-1.1.0.21\nbaseurl=http://10.161.0.15/HDP-UTILS/\n\npath=/\nenabled=1\ngpgcheck=0'}
2017-07-14 16:46:09,644 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2017-07-14 16:46:09,732 - Skipping installation of existing package unzip
2017-07-14 16:46:09,736 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2017-07-14 16:46:09,749 - Skipping installation of existing package curl
2017-07-14 16:46:09,749 - Package['hdp-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2017-07-14 16:46:09,762 - Skipping installation of existing package hdp-select
2017-07-14 16:46:09,975 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2017-07-14 16:46:09,983 - Stack Feature Version Info: stack_version=2.6, version=None, current_cluster_version=None -> 2.6
2017-07-14 16:46:09,986 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2017-07-14 16:46:09,992 - checked_call['rpm -q --queryformat '%{version}-%{release}' hdp-select | sed -e 's/\.el[0-9]//g''] {'stderr': -1}
2017-07-14 16:46:10,033 - checked_call returned (0, '2.3.2.0-2950', '')
2017-07-14 16:46:10,039 - Package['hadoop_2_3_2_0_2950'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2017-07-14 16:46:10,124 - Installing package hadoop_2_3_2_0_2950 ('/usr/bin/yum -d 0 -e 0 -y install hadoop_2_3_2_0_2950')
2017-07-14 16:46:25,425 - Execution of '/usr/bin/yum -d 0 -e 0 -y install hadoop_2_3_2_0_2950' returned 1. Error: Cannot find a valid baseurl for repo: base
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os error was
14: PYCURL ERROR 7 - "Failed to connect to 2604:1580:fe02:2::10: Network is unreachable"
2017-07-14 16:46:25,426 - Failed to install package hadoop_2_3_2_0_2950. Executing '/usr/bin/yum clean metadata'
2017-07-14 16:46:25,633 - Retrying to install package hadoop_2_3_2_0_2950 after 30 seconds
2017-07-14 16:47:10,887 - Could not find stack selector for stack: HDP
Command failed after 1 tries Looks like the issue is caused by the following:
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os error was
14: PYCURL ERROR 7 - "Failed to connect to 2604:1580:fe02:2::10: Network is unreachable"
2017-07-14 16:46:25,426 - Failed to install package hadoop_2_3_2_0_2950. Executing '/usr/bin/yum clean metadata' I suspect it to be a proxy issue. I have configured the proxy (http_proxy, and https_proxy) in .bash_profile, and disabled proxy for the local repository, using no_proxy. Any suggestions on how to resolve this issue? Thanks,
... View more
Labels: