Support Questions
Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Advanced Search

Does Spark need Python 3 and dependencies on all cluster nodes to work with Python 3 ?

Does Spark need Python 3 and dependencies on all cluster nodes to work with Python 3 ?

nyakkanti
Expert Contributor

Created ‎04-04-2018 03:13 AM

Does Spark need Python 3 and dependencies on all cluster nodes to work with Python 3 ?

Reply
380 Views
0 Kudos
1 REPLY 1
Highlighted

Re: Does Spark need Python 3 and dependencies on all cluster nodes to work with Python 3 ?

hkwon
New Contributor

Created ‎04-04-2018 05:38 AM

I believe PySpark doesn't guarantee working with different Python versions. It should be the best to match and have the same Python with driver side.

Reply
276 Views
0 Kudos
Don't have an account?
Register
Announcements
Product Announcements
[ANNOUNCE] New Cloudera JDBC 2.6.20 Driver for Apache Impala Released
Product Announcements
Transition to private repositories for CDH, HDP and HDF
Product Announcements
[ANNOUNCE] New Applied ML Research from Cloudera Fast Forward: Few-Shot Text Classification
Product Announcements
[ANNOUNCE] New JDBC 2.6.13 Driver for Apache Hive Released
Product Announcements
[ANNOUNCE] Refreshed Research from Cloudera Fast Forward: Semantic Image Search and Federated Learning
View More Announcements