Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!
Labels (1)
avatar

I recently received a question "We'd like to have Python and R installed in the docker image for a package.  Is it possible to have such customized docker with python >=3.7, R>=4.0 and CUDA==11.7?"

 

I'd like to go through how I accomplished this using the pbj-workbench-r4.1-standard.Dockerfile.  This runtime has the essentials (Python and R) except for CUDA (NVIDIA GPU).  Therefore, we'll use pbj-workbench-r4.1-standard.Dockerfile as a base image.  It's important to note that CML requires a runtime kernel that needs to be either R or Python, it can't be both.  I'll go through three different iterations of this image:

 

Iteration 1 - Since this image already has R and Python installed, we'll change the runtime NOT to use R

ryancicak/pbj_r-and-python:v9 -> Python 3.10 (GitHub Tag: python/3.10/standard_dockerhub_v9)
https://github.com/ryancicak/pbj_r-and-python/blob/python/3.10/standard_dockerhub_v9/Dockerfile

 

Iteration 2 - Go back to using R (as we did initially before removing R as the runtime kernel) in Iteration 1

ryancicak/pbj_r-and-python:v10 -> R 4.1 (GitHub Tag: r/4.1/standard_dockerhub_v10)
https://github.com/ryancicak/pbj_r-and-python/blob/r/4.1/standard_dockerhub_v10/Dockerfile

 

Iteration 3 - Searching the nvidia/cuda dockerhub tags, I found version 11.7.1, with similarities to 

https://github.com/cloudera/ml-runtimes/blob/public-runtimes/pbj-workbench-python3.10-cuda.Dockerfil...

:

 

# Copyright 2022 Cloudera. All Rights Reserved.
FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu20.04
RUN apt-key del 7fa2af80 && apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub

 

 to

 

# Copyright 2022 Cloudera. All Rights Reserved.
FROM nvidia/cuda:11.7.1-cudnn8-devel-ubuntu20.04
RUN apt-key del 7fa2af80 && apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub

 

*Notice "-cudnn8-devel-ubuntu20.04" is identical, as well as the following "RUN" line

iteration_3_cuda_11_7_running.png

*Validating cuda 11.7 is running

 

If we compare the changes between iteration 1 and 2, it's ever so slight when comparing tags on the Dockerfile.  Again, we're only changing the runtime kernel since the image has both R and Python installed.  Feel free to try these different iterations in the Docker Hub tags as specified within CML by adding these as new Runtimes:

 

Iteration 1 - ryancicak/pbj_r-and-python:v9

Iteration 2 - ryancicak/pbj_r-and-python:v10

Iteration 3 - ryancicak/pbj_r-and-python:v11

 

While the versioning is different in CML, they're all basically using the same docker image as asked in the original question above.  

 

Shout-out to @amarinovszki for his collaboration!  Thanks Árpád!

 

2,194 Views
0 Kudos
Version history
Last update:
‎04-20-2026 11:50 PM
Updated by: