Support Questions
Find answers, ask questions, and share your expertise

How To Pass Dataframe as argument to PySpark UDF

How To Pass Dataframe as argument to PySpark UDF

Contributor

Hi,

 

I have the below requirement, i need to itterate through each rows in dataframe against remaining rows and need to apply some transformation on it.

 

I tried using python function it is not applying to each row.

I tried UDF but it seems UDF will not accept Dataframe object as input

 

Is there any way to address my problem ?

 

Regards,

MJ