Reply
New Contributor
Posts: 4
Registered: ‎03-15-2018

engine tls error

Good day,

got a few issues with cdsw engine.

CDSW is behind corporate proxy. Corporate proxy uses self-signed certificate.

 

I've added corporate root CA to CDSW master (master has no issues with TLS).

 

Standard cdsw engine fails with tls erros. These tools aren't working - wget, curl, conda, pip, R (install.packages()), apt-get.

 

I've created a modified cdsw engine container. This container has config files to ignore TLS:

/etc/pip.conf

/etc/wgetrc

/etc/R/Renviron.site

 

/etc/curlrc and ~/.curlrc with "insecure" instruction is not honored, curl works only with -k option.

 

I've also tried to push corporate root CAs to cdsw engine container and issue "update-ca-roots --fresh". I see root CAs listed in /etc/ssl/certs/ca-certificates.crt, inside my custom cdsw engine. But still get TLS errors in tools mentioned above.

 

Could it be some kind of "clash" between corporate proxy and Traefik?

Or maybe too many redirects which causes requests python lib fails?

Am I correct to assume that cdsw user works in a conda virtual environment by default?

Is there an open repo with CDSW pods/engine configuration?

 

 

 

Conda errors:

1. Without condarc

cdsw@hdztfwk4fj9sm7xe:~$ conda update conda
PackageNotInstalledError: Package is not installed in prefix.
prefix: /home/cdsw/.conda/envs/python2.7
package name: conda

2. Without condarc

cdsw@hdztfwk4fj9sm7xe:~$ sudo conda update conda
sudo: conda: command not found

 

3.

cdsw@hdztfwk4fj9sm7xe:~$ conda install nltk
Fetching package metadata ...
CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://repo.continuum.io/pkgs/main/linux-64/repodata.json.bz2>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
SSLError(SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),),)


4. With ~/.condarc
#cat ~/.condarc
proxy_servers:
http: http://proxy:8080
https: http://proxy:8080
ssl_verify: False

cdsw@hdztfwk4fj9sm7xe:~$ conda install nltk
Fetching package metadata ...An unexpected error has occurred.
Please consider posting the following information to the
conda GitHub issue tracker at:
https://github.com/conda/conda/issues
Current conda install:
platform : linux-64
conda version : 4.3.30
conda is private : False
conda-env version : 4.3.30
conda-build version : not installed
python version : 2.7.13.final.0
requests version : 2.12.4
root environment : /opt/conda (read only)
default environment : /home/cdsw/.conda/envs/python2.7
envs directories : /home/cdsw/.conda/envs
/opt/conda/envs
package cache : /opt/conda/pkgs
/home/cdsw/.conda/pkgs
channel URLs : https://repo.continuum.io/pkgs/main/linux-64
https://repo.continuum.io/pkgs/main/noarch
https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/r/linux-64
https://repo.continuum.io/pkgs/r/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : /home/cdsw/.condarc
netrc file : None
offline mode : False
user-agent : conda/4.3.30 requests/2.12.4 CPython/2.7.13 Linux/3.10.0-693.el7.x86_64 debian/stretch/sid glibc/2.23 
UID:GID : 8536:8536
`$ /opt/conda/bin/conda install nltk`
Traceback (most recent call last):
File "/opt/conda/lib/python2.7/site-packages/conda/exceptions.py", line 640, in conda_exception_handler
return_value = func(*args, **kwargs)
File "/opt/conda/lib/python2.7/site-packages/conda/cli/main.py", line 140, in _main
exit_code = args.func(args, p)
File "/opt/conda/lib/python2.7/site-packages/conda/cli/main_install.py", line 80, in execute
install(args, parser, 'install')
File "/opt/conda/lib/python2.7/site-packages/conda/cli/install.py", line 231, in install
unknown=index_args['unknown'], prefix=prefix)
File "/opt/conda/lib/python2.7/site-packages/conda/core/index.py", line 101, in get_index
index = fetch_index(channel_priority_map, use_cache=use_cache)
File "/opt/conda/lib/python2.7/site-packages/conda/core/index.py", line 120, in fetch_index
repodatas = collect_all_repodata(use_cache, tasks)
File "/opt/conda/lib/python2.7/site-packages/conda/core/repodata.py", line 75, in collect_all_repodata
repodatas = _collect_repodatas_serial(use_cache, tasks)
File "/opt/conda/lib/python2.7/site-packages/conda/core/repodata.py", line 485, in _collect_repodatas_serial
for url, schan, pri in tasks]
File "/opt/conda/lib/python2.7/site-packages/conda/core/repodata.py", line 115, in func
res = f(*args, **kwargs)
File "/opt/conda/lib/python2.7/site-packages/conda/core/repodata.py", line 464, in fetch_repodata
mod_etag_headers.get('_mod'))
File "/opt/conda/lib/python2.7/site-packages/conda/core/repodata.py", line 156, in fetch_repodata_remote_request
json_str = maybe_decompress(filename, resp.content)
File "/opt/conda/lib/python2.7/site-packages/conda/core/repodata.py", line 154, in maybe_decompress
if filename.endswith('.bz2')
IOError: invalid data stream

 

5. With /etc/condarc
#cat /etc/condarc
proxy_servers:
http: http://proxy:8080
https: http://proxy:8080
ssl_verify: False

 

cdsw@hdztfwk4fj9sm7xe:~$ conda install nltk
Fetching package metadata ...
CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://repo.continuum.io/pkgs/main/linux-64/repodata.json.bz2>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
SSLError(SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),),)

 

 

P.S. Feature request - sudo by default in a default engine.

New Contributor
Posts: 4
Registered: ‎03-15-2018

Re: engine tls error

Added ssl_verify: /etc/ssl/certs/ca-certificates.crt to /opt/conda/.condarc (ca-certificates.crt has private root CA for proxy server)

 

and tried

 

#conda install -y -c conda-forge python=2.7.11 feather-format

 

Got the same error:

Fetching package metadata .....An unexpected error has occurred.
Please consider posting the following information to the
conda GitHub issue tracker at:
    https://github.com/conda/conda/issues
Current conda install:
               platform : linux-64
          conda version : 4.3.30
       conda is private : False
      conda-env version : 4.3.30
    conda-build version : not installed
         python version : 2.7.13.final.0
       requests version : 2.12.4
       root environment : /opt/conda  (read only)
    default environment : /home/cdsw/.conda/envs/python2.7
       envs directories : /home/cdsw/.conda/envs
                          /opt/conda/envs
          package cache : /opt/conda/pkgs
                          /home/cdsw/.conda/pkgs
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.continuum.io/pkgs/main/linux-64
                          https://repo.continuum.io/pkgs/main/noarch
                          https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
            config file : /opt/conda/.condarc
             netrc file : None
           offline mode : False
             user-agent : conda/4.3.30 requests/2.12.4 CPython/2.7.13 Linux/3.10.0-693.el7.x86_64 debian/stretch/sid glibc/2.23    
                UID:GID : 8536:8536
    Traceback (most recent call last):
`$ /opt/conda/bin/conda install -y -c conda-forge python=2.7.11 feather-format`




    Traceback (most recent call last):
      File "/opt/conda/lib/python2.7/site-packages/conda/exceptions.py", line 640, in conda_exception_handler
        return_value = func(*args, **kwargs)
      File "/opt/conda/lib/python2.7/site-packages/conda/cli/main.py", line 140, in _main
        exit_code = args.func(args, p)
      File "/opt/conda/lib/python2.7/site-packages/conda/cli/main_install.py", line 80, in execute
        install(args, parser, 'install')
      File "/opt/conda/lib/python2.7/site-packages/conda/cli/install.py", line 231, in install
        unknown=index_args['unknown'], prefix=prefix)
      File "/opt/conda/lib/python2.7/site-packages/conda/core/index.py", line 101, in get_index
        index = fetch_index(channel_priority_map, use_cache=use_cache)
      File "/opt/conda/lib/python2.7/site-packages/conda/core/index.py", line 120, in fetch_index
        repodatas = collect_all_repodata(use_cache, tasks)
      File "/opt/conda/lib/python2.7/site-packages/conda/core/repodata.py", line 75, in collect_all_repodata
        repodatas = _collect_repodatas_serial(use_cache, tasks)
      File "/opt/conda/lib/python2.7/site-packages/conda/core/repodata.py", line 485, in _collect_repodatas_serial
        for url, schan, pri in tasks]
      File "/opt/conda/lib/python2.7/site-packages/conda/core/repodata.py", line 115, in func
        res = f(*args, **kwargs)
      File "/opt/conda/lib/python2.7/site-packages/conda/core/repodata.py", line 464, in fetch_repodata
        mod_etag_headers.get('_mod'))
      File "/opt/conda/lib/python2.7/site-packages/conda/core/repodata.py", line 156, in fetch_repodata_remote_request
        json_str = maybe_decompress(filename, resp.content)
      File "/opt/conda/lib/python2.7/site-packages/conda/core/repodata.py", line 154, in maybe_decompress
        if filename.endswith('.bz2')
    IOError: invalid data stream

Will try to update conda version and see if there's a bug in the current one.

Announcements