Support Questions

wonhee_choi · ‎07-14-2016

Hi All,

Zeppelin installation via Ambari fails on HDP2.4. RHEL6.5 64bit.

Below are the installation logs from Ambari. The yum command in the log actually fails when run manually.

Manually installed pip as below but when uninstalled and installed again the same error.

[root@ip-172-31-24-221 ~]# curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1488k  100 1488k    0     0  19.1M      0 --:--:-- --:--:-- --:--:-- 45.4M
[root@ip-172-31-24-221 ~]# 
[root@ip-172-31-24-221 ~]# python get-pip.py
DEPRECATION: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. A future version of pip will drop support for Python 2.6
Collecting pip
/tmp/tmpfrM4hF/pip.zip/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
/tmp/tmpfrM4hF/pip.zip/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  Downloading pip-8.1.2-py2.py3-none-any.whl (1.2MB)
    100% |████████████████████████████████| 1.2MB 978kB/s 
Collecting wheel
  Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)
    100% |████████████████████████████████| 71kB 9.6MB/s 
Collecting argparse; python_version == "2.6" (from wheel)
  Downloading argparse-1.4.0-py2.py3-none-any.whl
Installing collected packages: pip, argparse, wheel
  Found existing installation: argparse 1.2.1
    Uninstalling argparse-1.2.1:
      Successfully uninstalled argparse-1.2.1
Successfully installed argparse-1.4.0 pip-8.1.2 wheel-0.29.0
/tmp/tmpfrM4hF/pip.zip/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
[root@ip-172-31-24-221 ~]# pip -V
pip 8.1.2 from /usr/lib/python2.6/site-packages (python 2.6)

How could I get around this problem and install Zeppelin please?

Plus, after this failure, starting all services in Ambari stopped working. I have to start/stop each service at a time. It is not clear whether Zeppelin install failure somehow affect this.

Please advise.

Thanks,

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.4/services/ZEPPELIN/package/scripts/master.py", line 235, in <module>
    Master().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.4/services/ZEPPELIN/package/scripts/master.py", line 54, in install
    self.install_packages(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 404, in install_packages
    Package(name)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 158, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 121, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 49, in action_install
    self.install_package(package_name, self.resource.use_repos, self.resource.skip_repos)
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 49, in install_package
    shell.checked_call(cmd, sudo=True, logoutput=self.get_logoutput())
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
    tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
    raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum -d 0 -e 0 -y install python-pip' returned 1. Error: Nothing to do

2016-07-14 10:18:17,167 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.4.2.0-258
2016-07-14 10:18:17,167 - Checking if need to create versioned conf dir /etc/hadoop/2.4.2.0-258/0
2016-07-14 10:18:17,167 - call['conf-select create-conf-dir --package hadoop --stack-version 2.4.2.0-258 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1}
2016-07-14 10:18:17,189 - call returned (1, '/etc/hadoop/2.4.2.0-258/0 exist already', '')
2016-07-14 10:18:17,189 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.4.2.0-258 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False}
2016-07-14 10:18:17,210 - checked_call returned (0, '')
2016-07-14 10:18:17,211 - Ensuring that hadoop has the correct symlink structure
2016-07-14 10:18:17,211 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-07-14 10:18:17,212 - Group['spark'] {}
2016-07-14 10:18:17,214 - Group['zeppelin'] {}
2016-07-14 10:18:17,214 - Adding group Group['zeppelin']
2016-07-14 10:18:17,238 - Group['hadoop'] {}
2016-07-14 10:18:17,239 - Group['users'] {}
2016-07-14 10:18:17,239 - Group['knox'] {}
2016-07-14 10:18:17,239 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,240 - User['storm'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,241 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,241 - User['oozie'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']}
2016-07-14 10:18:17,242 - User['atlas'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,243 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,243 - User['falcon'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']}
2016-07-14 10:18:17,244 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']}
2016-07-14 10:18:17,244 - User['zeppelin'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,245 - Adding user User['zeppelin']
2016-07-14 10:18:17,296 - User['spark'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,297 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']}
2016-07-14 10:18:17,297 - User['flume'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,298 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,299 - User['sqoop'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,299 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,300 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,301 - User['hbase'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,301 - User['knox'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,302 - User['hcat'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-07-14 10:18:17,303 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2016-07-14 10:18:17,305 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2016-07-14 10:18:17,309 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2016-07-14 10:18:17,309 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'recursive': True, 'mode': 0775, 'cd_access': 'a'}
2016-07-14 10:18:17,310 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2016-07-14 10:18:17,311 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
2016-07-14 10:18:17,314 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] due to not_if
2016-07-14 10:18:17,315 - Group['hdfs'] {}
2016-07-14 10:18:17,315 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'hdfs']}
2016-07-14 10:18:17,316 - Directory['/etc/hadoop'] {'mode': 0755}
2016-07-14 10:18:17,331 - File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2016-07-14 10:18:17,332 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 0777}
2016-07-14 10:18:17,345 - Repository['HDP-2.4'] {'base_url': 'http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.4.2.0', 'action': ['create'], 'components': ['HDP', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP', 'mirror_list': None}
2016-07-14 10:18:17,354 - File['/etc/yum.repos.d/HDP.repo'] {'content': '[HDP-2.4]\nname=HDP-2.4\nbaseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.4.2.0\n\npath=/\nenabled=1\ngpgcheck=0'}
2016-07-14 10:18:17,354 - Repository['HDP-UTILS-1.1.0.20'] {'base_url': 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6', 'action': ['create'], 'components': ['HDP-UTILS', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS', 'mirror_list': None}
2016-07-14 10:18:17,358 - File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content': '[HDP-UTILS-1.1.0.20]\nname=HDP-UTILS-1.1.0.20\nbaseurl=http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6\n\npath=/\nenabled=1\ngpgcheck=0'}
2016-07-14 10:18:17,358 - Package['unzip'] {}
2016-07-14 10:18:17,460 - Skipping installation of existing package unzip
2016-07-14 10:18:17,460 - Package['curl'] {}
2016-07-14 10:18:17,499 - Skipping installation of existing package curl
2016-07-14 10:18:17,499 - Package['hdp-select'] {}
2016-07-14 10:18:17,537 - Skipping installation of existing package hdp-select
2016-07-14 10:18:17,699 - Execute['find /var/lib/ambari-agent/cache/stacks/HDP/2.4/services/ZEPPELIN/package -iname "*.sh" | xargs chmod +x'] {}
2016-07-14 10:18:17,708 - Execute['echo platform.linux_distribution:Red Hat Enterprise Linux Server+6.5+Santiago'] {}
2016-07-14 10:18:17,712 - Package['gcc-gfortran'] {}
2016-07-14 10:18:17,812 - Installing package gcc-gfortran ('/usr/bin/yum -d 0 -e 0 -y install gcc-gfortran')
2016-07-14 10:18:22,975 - Package['blas-devel'] {}
2016-07-14 10:18:23,016 - Installing package blas-devel ('/usr/bin/yum -d 0 -e 0 -y install blas-devel')
2016-07-14 10:18:25,378 - Package['lapack-devel'] {}
2016-07-14 10:18:25,418 - Installing package lapack-devel ('/usr/bin/yum -d 0 -e 0 -y install lapack-devel')
2016-07-14 10:18:29,415 - Package['python-devel'] {}
2016-07-14 10:18:29,458 - Skipping installation of existing package python-devel
2016-07-14 10:18:29,459 - Package['python-pip'] {}
2016-07-14 10:18:29,497 - Installing package python-pip ('/usr/bin/yum -d 0 -e 0 -y install python-pip')

wonhee_choi · ‎07-14-2016

Resolved.

I followed the below document to install python-pip manually in another way:

http://www.liquidweb.com/kb/how-to-install-pip-on-centos-7/

[root@ip-172-31-25-3 ~]# wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
--2016-07-14 11:49:14--  https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
Resolving dl.fedoraproject.org... 209.132.181.25, 209.132.181.26, 209.132.181.27, ...
Connecting to dl.fedoraproject.org|209.132.181.25|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14540 (14K) [application/x-rpm]
Saving to: “epel-release-latest-6.noarch.rpm”

100%[======================================================================================================================================>] 14,540      90.8K/s   in 0.2s    

2016-07-14 11:49:15 (90.8 KB/s) - “epel-release-latest-6.noarch.rpm” saved [14540/14540]

[root@ip-172-31-25-3 ~]# ll
total 20
-rw-r--r-- 1 root root 14540 Nov  6  2012 epel-release-latest-6.noarch.rpm
drwxr-xr-x 3 root root  4096 May 30 13:11 guardium_work
[root@ip-172-31-25-3 ~]# sudo rpm -Uvh epel-release-latest-6.noarch.rpm
warning: epel-release-latest-6.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Preparing...                ########################################### [100%]
   1:epel-release           ########################################### [100%]
[root@ip-172-31-25-3 ~]# yum -y install python-pip
Loaded plugins: amazon-id, rhui-lb, security
epel/metalink                                                                                                                                            | 2.7 kB     00:00     
epel                                                                                                                                                     | 4.3 kB     00:00     
epel/primary_db                                                                                                                                          | 5.8 MB     00:00     
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package python-pip.noarch 0:7.1.0-1.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================================================================================================================
 Package                                     Arch                                    Version                                        Repository                             Size
================================================================================================================================================================================
Installing:
 python-pip                                  noarch                                  7.1.0-1.el6                                    epel                                  1.5 M

Transaction Summary
================================================================================================================================================================================
Install       1 Package(s)

Total download size: 1.5 M
Installed size: 6.6 M
Downloading Packages:
python-pip-7.1.0-1.el6.noarch.rpm                                                                                                                        | 1.5 MB     00:00     
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
Importing GPG key 0x0608B895:
 Userid : EPEL (6) <epel@fedoraproject.org>
 Package: epel-release-6-8.noarch (installed)
 From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
  Installing : python-pip-7.1.0-1.el6.noarch                                                                                                                                1/1 
  Verifying  : python-pip-7.1.0-1.el6.noarch                                                                                                                                1/1 

Installed:
  python-pip.noarch 0:7.1.0-1.el6                                                                                                                                               

Complete!
[root@ip-172-31-25-3 ~]#

After installing python-pip via yum, reinstalled zeppelin and it went fine.

Hope this helps.

View solution in original post

wonhee_choi · ‎07-14-2016