I have pyspark script(p1) which have dataframes created and returns dataframe. the same is being imported into different python script(p2). when i run p1 directly script executes successfully however when i run p2 it failes saying "no module found p1". I have import p1 into p2 script.
Just to let you know, scripts works if we do not have Spark commands in script which is being imported to other script. So it looks like i am missing something to include.