Skip to main content
DevOpsWorld
Bronze III
Aug 19, 2026 12:08 PM UTC
Question

How to enable Spark 3.5 as a Runtime Addon for Cloudera AI Workbench sessions?

Hi all,

We've upgraded our base cluster (Cloudera Runtime 7.1.9) to CDS 3.5
via the new add-on parcel — that part is working fine.

Now we're trying to make Spark 3.5 available as a selectable Runtime
Addon inside Cloudera AI Workbench sessions, but we're stuck.

Our setup:
- Cloudera AI Workbench version: 2.0.44-h1-b4
- CDP Private Cloud Data Services: 1.5.4-h10-b2
- Deployment type: ECS
- We use Spark Pushdown (sessions submit jobs that run on the base
cluster via YARN)

What we've found so far:
- Currently only Spark versions up to 3.3.2 show up in Site
Administration > Runtime > Runtime Addons
- There's a SPARK3 3.5.4 parcel visible in Cloudera Manager under
our CAI cluster, listed as "Available Remotely" — not yet
downloaded/activated

Has anyone gotten Spark 3.5 working as a Runtime Addon on a similar
setup? Specifically:
1. Is downloading/activating that SPARK3 parcel the right approach,
or does it need to be "linked" to the workbench some other way?
2. Does this require a minimum Workbench version?
3. Any pushdown-specific gotchas with Spark 3.5?

Thanks in advance!

2 replies

vafs
Gold ⭐️⭐️⭐️
Gold ⭐️⭐️⭐️
Aug 19, 2026 12:29 PM UTC

Hello @DevOpsWorld, 

Thanks for reaching out on the community. It's always a pleasure. 

The problem here is the DS version you have. Data Services 1.5.4 only supports Spark 2.4.7 and 3.2.3: 

https://docs.cloudera.com/machine-learning/1.5.4/spark/topics/ml-spark-supported-versions.html

To be able to use Spark 3.5 you need to upgrade to DS 1.5.5: 

https://docs.cloudera.com/machine-learning/1.5.5/spark/topics/ml-spark-supported-versions.html

Hope this helps to clarify the situation. 

Translate
DianaTorres
Community Manager
Community Manager
Aug 24, 2026 11:12 PM UTC

@DevOpsWorld Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  Thanks.

Translate
Regards, Diana Torres, Senior Community Moderator