Member since
07-06-2018
24
Posts
4
Kudos Received
4
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 4896 | 11-06-2019 04:11 AM |
11-06-2019
04:11 AM
After we configured these firewall rules, I was able to build the custom image: http://archive.cloudera.com/* PORT: 80 http://ppa.launchpad.net/* PORT: 80 Hint on docker company-registry: I was at the beginning a bit confused about the docker company-registry because we never set up a public docker registry. This is important because when you build and push the docker image, the commands require the company-registry. I could find it simply with the command docker images You will get a list of images registered on your machine. I found out that the company registry in my case was docker.repository.cloudera.com/cdsw/ So what I did was just building the docker image with my company-registry. In my case there was no need to push the image to docker. (see CDSW GPU Guide) docker build --network host -t docker.repository.cloudera.com/cdsw/cdsw-cuda:8 . -f cuda.Dockerfile After that, the image was built successfully. The image can be listed with the command docker images One should see the newly created docker image in the repository. For the last step the site admin. has to add this image to CDSW (see CDSW GPU Guide). The Repository:Tag in CDSW would be the same as shown above with the company-registry. In this example it would be Repository:Tag docker.repository.cloudera.com/cdsw/cdsw-cuda:8 For airgapped installations you need these firewall rules to be able to build the NVIDIA image for CDSW: (all on Port 80) http://developer.download.nvidia.com/* http://archive.ubuntu.com/ubuntu/* http://security.ubuntu.com/ubuntu/* http://archive.cloudera.com/* http://ppa.launchpad.net/* Regards.
... View more
10-17-2019
08:43 AM
Happy to hear that. Yes, we have this in Radar and soon our doc will be updated with known issue in 1.6 version. Cheers, Keep visiting and contributing Cloudera Community
... View more
10-15-2019
04:21 PM
Thank you very much for the reply and answer @Bhuv
... View more
04-05-2019
01:03 AM
1 Kudo
Thank you for the great explanation @AutoIN. This solved my problem. On our CDSW cluster we have 2 nodes with a master and a slave. As described, I was able to figure out that the available cpu and memeory on both hosts are badly distributed. As an example, I'm able to spin an engine with a lot of vcpus but with little memory and vice versa. I was just not aware that a session can't share resources across nodes. Thank you very much!
... View more