Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Need clarification on role of Catalyst optimizer and Project Tungsten

avatar
New Contributor

Need clarification on role of Catalyst optimizer and Project Tungsten

Catalyst optimizer will produce optimized Physical plan from logical plan.

The optimized physical plan will then taken by Code generator to emit Rdd's.

Here the Code generator is part of Project Tungsten or Catalyst Optimizer?

Here the Code generator is called as Whole Stage Code generator?

When I look at lot of article Code generator will emit Rdd.

Who will then generate bytecode from the RDD?

I am totally confused with Code generator and Project Tungsten. Please help me

1 REPLY 1

avatar
Expert Contributor