Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cannot get graphframes to work in pyspark2

Highlighted

Cannot get graphframes to work in pyspark2

New Contributor

Does anyone know what the procedure is for installing graphframes for pyspark2 on 

SPARK2-2.0.0.cloudera1-1.cdh5.7.0.p0.113931?

 

Or more generally, how to instal any package for spark in cloudera?.

 

1 REPLY 1
Highlighted

Re: Cannot get graphframes to work in pyspark2

Expert Contributor

You can use "--packages" with the shell to include any additional 3rd party packages you'd want. For graphframes , please refer to https://spark-packages.org/package/graphframes/graphframes, pick the suitable spark version and install it.
 
Since you are on Spark ver 2.0 and assuming the scala ver 2.11, you should likely use 0.5.0-spark2.0-s_2.11

 

# pyspark2 --packages graphframes:graphframes:0.5.0-spark2.0-s_2.11
Don't have an account?
Coming from Hortonworks? Activate your account here