Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st. We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here. Need help or have questions? Drop us a line at [email protected]
Created on 07-27-202310:57 AM - edited on 04-20-202611:50 PM by GrazittiAPI
I recently received a question "We'd like to have Python and R installed in the docker image for a package. Is it possible to have such customized docker with python >=3.7, R>=4.0 and CUDA==11.7?"
I'd like to go through how I accomplished this using the pbj-workbench-r4.1-standard.Dockerfile. This runtime has the essentials (Python and R) except for CUDA (NVIDIA GPU). Therefore, we'll use pbj-workbench-r4.1-standard.Dockerfile as a base image. It's important to note that CML requires a runtime kernel that needs to be either R or Python, it can't be both. I'll go through three different iterations of this image:
Iteration 1 - Since this image already has R and Python installed, we'll change the runtime NOT to use R
# Copyright 2022 Cloudera. All Rights Reserved.
FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu20.04
RUN apt-key del 7fa2af80 && apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub
to
# Copyright 2022 Cloudera. All Rights Reserved.
FROM nvidia/cuda:11.7.1-cudnn8-devel-ubuntu20.04
RUN apt-key del 7fa2af80 && apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub
*Notice "-cudnn8-devel-ubuntu20.04" is identical, as well as the following "RUN" line
*Validating cuda 11.7 is running
If we compare the changes between iteration 1 and 2, it's ever so slight when comparing tags on the Dockerfile. Again, we're only changing the runtime kernel since the image has both R and Python installed. Feel free to try these different iterations in the Docker Hub tags as specified within CML by adding these as new Runtimes:
Iteration 1 - ryancicak/pbj_r-and-python:v9
Iteration 2 - ryancicak/pbj_r-and-python:v10
Iteration 3 - ryancicak/pbj_r-and-python:v11
While the versioning is different in CML, they're all basically using the same docker image as asked in the original question above.
Shout-out to @amarinovszki for his collaboration! Thanks Árpád!