Support Questions
Find answers, ask questions, and share your expertise

Need clarification on role of Catalyst optimizer and Project Tungsten

New Contributor

Need clarification on role of Catalyst optimizer and Project Tungsten

Catalyst optimizer will produce optimized Physical plan from logical plan.

The optimized physical plan will then taken by Code generator to emit Rdd's.

Here the Code generator is part of Project Tungsten or Catalyst Optimizer?

Here the Code generator is called as Whole Stage Code generator?

When I look at lot of article Code generator will emit Rdd.

Who will then generate bytecode from the RDD?

I am totally confused with Code generator and Project Tungsten. Please help me

1 REPLY 1

Expert Contributor
; ;