Thanks for your reply, but it seems your script doesn't work The dataset delimiter is shift-out (\x0f) and line-separator is shift-in (\x0e) in pandas, i can simply load the data into dataframe using this command: df1 = pd.read_csv("/folder/file.gz", sep = '\x0f', lineterminator = '\x0e' ) May I know how to do this in spark?
... View more
