Reply
New Contributor
Posts: 2
Registered: ‎05-09-2018

Apache Spark "Cost Based Optimization" not supported.

I recently discovered that CDH does not support Spark 2's new Cost Based SQL Optimizer.

 

Config: spark.sql.cbo.enabled

Descriptionhttps://databricks.com/blog/2017/08/31/cost-based-optimizer-in-apache-spark-2-2.html 

Non-Support Statementhttps://www.cloudera.com/documentation/spark2/latest/topics/spark2_known_issues.html#ki_cbo_unsuppor...

 

This is very disappointing, could a Cloudera staff member please explain why, and to what extent this feature is not supported?

Announcements