Support Questions
Find answers, ask questions, and share your expertise

Cannot get graphframes to work in pyspark2

Cannot get graphframes to work in pyspark2

New Contributor

Does anyone know what the procedure is for installing graphframes for pyspark2 on 

SPARK2-2.0.0.cloudera1-1.cdh5.7.0.p0.113931?

 

Or more generally, how to instal any package for spark in cloudera?.

 

1 REPLY 1

Re: Cannot get graphframes to work in pyspark2

Expert Contributor

You can use "--packages" with the shell to include any additional 3rd party packages you'd want. For graphframes , please refer to https://spark-packages.org/package/graphframes/graphframes, pick the suitable spark version and install it.
 
Since you are on Spark ver 2.0 and assuming the scala ver 2.11, you should likely use 0.5.0-spark2.0-s_2.11

 

# pyspark2 --packages graphframes:graphframes:0.5.0-spark2.0-s_2.11