Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

PySpark: Broadcast Java objects

Highlighted

PySpark: Broadcast Java objects

Contributor

I'm experimenting with broadcast variables in PySpark at the moment, and I've noticed that whenever I create an explicit Java object using sc._jvm, I get errors when I try to broadcast these variables. Looking at the stack trace, the problem seems to be related to pickling. Does anybody know how I can broadcast such variables?

Don't have an account?
Coming from Hortonworks? Activate your account here