Created on 07-27-202310:57 AM - edited 07-27-202311:02 AM
I recently received a question "We'd like to have Python and R installed in the docker image for a package. Is it possible to have such customized docker with python >=3.7, R>=4.0 and CUDA==11.7?"
I'd like to go through how I accomplished this using the pbj-workbench-r4.1-standard.Dockerfile. This runtime has the essentials (Python and R) except for CUDA (NVIDIA GPU). Therefore, we'll use pbj-workbench-r4.1-standard.Dockerfile as a base image. It's important to note that CML requires a runtime kernel that needs to be either R or Python, it can't be both. I'll go through three different iterations of this image:
Iteration 1 - Since this image already has R and Python installed, we'll change the runtime NOT to use R
# Copyright 2022 Cloudera. All Rights Reserved.
FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu20.04
RUN apt-key del 7fa2af80 && apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub
to
# Copyright 2022 Cloudera. All Rights Reserved.
FROM nvidia/cuda:11.7.1-cudnn8-devel-ubuntu20.04
RUN apt-key del 7fa2af80 && apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub
*Notice "-cudnn8-devel-ubuntu20.04" is identical, as well as the following "RUN" line
*Validating cuda 11.7 is running
If we compare the changes between iteration 1 and 2, it's ever so slight when comparing tags on the Dockerfile. Again, we're only changing the runtime kernel since the image has both R and Python installed. Feel free to try these different iterations in the Docker Hub tags as specified within CML by adding these as new Runtimes:
Iteration 1 - ryancicak/pbj_r-and-python:v9
Iteration 2 - ryancicak/pbj_r-and-python:v10
Iteration 3 - ryancicak/pbj_r-and-python:v11
While the versioning is different in CML, they're all basically using the same docker image as asked in the original question above.
Shout-out to @amarinovszki for his collaboration! Thanks Árpád!