Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
avatar

I recently received a question "We'd like to have Python and R installed in the docker image for a package.  Is it possible to have such customized docker with python >=3.7, R>=4.0 and CUDA==11.7?"

 

I'd like to go through how I accomplished this using the pbj-workbench-r4.1-standard.Dockerfile.  This runtime has the essentials (Python and R) except for CUDA (NVIDIA GPU).  Therefore, we'll use pbj-workbench-r4.1-standard.Dockerfile as a base image.  It's important to note that CML requires a runtime kernel that needs to be either R or Python, it can't be both.  I'll go through three different iterations of this image:

 

Iteration 1 - Since this image already has R and Python installed, we'll change the runtime NOT to use R

ryancicak/pbj_r-and-python:v9 -> Python 3.10 (GitHub Tag: python/3.10/standard_dockerhub_v9)
https://github.com/ryancicak/pbj_r-and-python/blob/python/3.10/standard_dockerhub_v9/Dockerfile

 

Iteration 2 - Go back to using R (as we did initially before removing R as the runtime kernel) in Iteration 1

ryancicak/pbj_r-and-python:v10 -> R 4.1 (GitHub Tag: r/4.1/standard_dockerhub_v10)
https://github.com/ryancicak/pbj_r-and-python/blob/r/4.1/standard_dockerhub_v10/Dockerfile

 

Iteration 3 - Searching the nvidia/cuda dockerhub tags, I found version 11.7.1, with similarities to 

https://github.com/cloudera/ml-runtimes/blob/public-runtimes/pbj-workbench-python3.10-cuda.Dockerfil...

:

 

# Copyright 2022 Cloudera. All Rights Reserved.
FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu20.04
RUN apt-key del 7fa2af80 && apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub

 

 to

 

# Copyright 2022 Cloudera. All Rights Reserved.
FROM nvidia/cuda:11.7.1-cudnn8-devel-ubuntu20.04
RUN apt-key del 7fa2af80 && apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub

 

*Notice "-cudnn8-devel-ubuntu20.04" is identical, as well as the following "RUN" line

iteration_3_cuda_11_7_running.png

*Validating cuda 11.7 is running

 

If we compare the changes between iteration 1 and 2, it's ever so slight when comparing tags on the Dockerfile.  Again, we're only changing the runtime kernel since the image has both R and Python installed.  Feel free to try these different iterations in the Docker Hub tags as specified within CML by adding these as new Runtimes:

 

Iteration 1 - ryancicak/pbj_r-and-python:v9

Iteration 2 - ryancicak/pbj_r-and-python:v10

Iteration 3 - ryancicak/pbj_r-and-python:v11

 

While the versioning is different in CML, they're all basically using the same docker image as asked in the original question above.  

 

Shout-out to @amarinovszki for his collaboration!  Thanks Árpád!

 

411 Views
0 Kudos
webinar banner
Version history
Last update:
‎07-27-2023 11:02 AM
Updated by:
meetups banner