Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Adding customized runtime into CDSW

avatar
Rising Star

I tried steps below. 

1. Installed harbor. (aaa.bbb.ccc)

2. Created new image use the dockerfile and command below

--------

FROM aaa.bbb.ccc/cloudera/cdsw/ml-runtime-jupyterlab-python3.10-standard:2023.08.2-b8 

RUN apt-get update && apt-get upgrade -y && apt-get clean && rm -rf /var/lib/apt/lists/

# Cloudera metadata for CML
ENV ML_RUNTIME_EDITION="EDITED RUNTIME" \
ML_RUNTIME_SHORT_VERSION="0.1" \
ML_RUNTIME_MAINTENANCE_VERSION=1 \
ML_RUNTIME_DESCRIPTION="EDITED RUNTIME to test"
ENV ML_RUNTIME_FULL_VERSION="${ML_RUNTIME_SHORT_VERSION}.${ML_RUNTIME_MAINTENANCE_VERSION}"
LABEL com.cloudera.ml.runtime.edition=$ML_RUNTIME_EDITION \
com.cloudera.ml.runtime.full-version=$ML_RUNTIME_FULL_VERSION \
com.cloudera.ml.runtime.short-version=$ML_RUNTIME_SHORT_VERSION \
com.cloudera.ml.runtime.maintenance-version=$ML_RUNTIME_MAINTENANCE_VERSION \
com.cloudera.ml.runtime.description=$ML_RUNTIME_DESCRIPTION

---------

docker build -t aaa.bbb.ccc/cloudera/cdsw/ml-runtime-jupyterlab-python3.10-standard:2023.08.2-b8-20240509 .

push aaa.bbb.ccc/cloudera/cdsw/ml-runtime-jupyterlab-python3.10-standard:2023.08.2-b8-20240509

3. Created secret

kubectl create secret docker-registry regcred --docker-server=aaa.bbb.ccc --docker-username=admin --docker-password=PASSWORD -n default

4. Tried adding the runtime image 

[Runtime Catalog] - [+Add Runtime]

put "aaa.bbb.ccc/cloudera/cdsw/ml-runtime-jupyterlab-python3.10-standard:2023.08.2-b8-20240509" 

and push [validate] then i faced the message

"Could not fetch the image metadata."

 

Which procedure did you miss?

To use private registry without using such as harbor, How can I push into it?

1 REPLY 1

avatar
Rising Star

I have got the reason why because of certification problem of runtime manager pod.

fetch image metadata failed data = {"err":"parseImageSourceFailed : error pinging docker registry harbor.hadoop.com: Get \"https://harbor.hadoop.com/v2/\": x509: certificate signed by unknown authority","url":"harbor.hadoop.com/cloudera/cdsw/ml-runtime-jupyterlab-python3.11-standard:2024.02.1-b4"}

 

CA already registered on system. 

I created the secret "regcred".

I put the file "k8s-secret-regcred.yaml" on /etc/cdsw/patches.

I entered CA into Root CA Configuration of [Site-administration] - [Security]

 

How can I solve this issue?