Link Analysis using Spark Python

Stewart12586 — Fri, 16 Sep 2022 10:35:54 GMT

Hi, I need to create some graphs using PySpark to elaborate some link analysis research. I already see this link: http://kukuruku.co/hub/algorithms/social-network-analysis-spark-graphx But this algorithm is implemented in Scala which is very more complex to understand. Anyone have an idea on a white paper or some tutorial that do some link analysis research using PySpark? Thanks!

Re: Link Analysis using Spark Python

bwalter1 — Mon, 22 Aug 2016 18:55:39 GMT

Hi Pedro,

python API for Spark is still missing, however there is a git project with a higher level API on top of Spark GraphX called GraphFrames: (GraphFrames) . The project claims: "GraphX is to RDDs as GraphFrames are to DataFrames."

I haven't worked with it, however a quick test of their samples with Spark 1.6.2 worked:

Use pyspark like this:

pyspark --packages graphframes:graphframes:0.2.0-spark1.6-s_2.10

or use zeppelin and add the dependencies to the interpreter configuration.

Maybe this library has what you need.

question Re: Link Analysis using Spark Python in Archives of Support Questions (Read Only)

Link Analysis using Spark Python

Re: Link Analysis using Spark Python