Member since
02-09-2023
2
Posts
1
Kudos Received
0
Solutions
02-09-2023
06:29 AM
1 Kudo
Hi Steven, Even if my data is like this, its causing issue. "1st Data"|"2nd ""\P"" data"|"3rd data"|"4th data" What is causing issue? Any Idea. I know spark is having default escape as backslash. But why it is behaving like this.
... View more
02-09-2023
03:45 AM
I'm facing weird issue, not sure why Spark is behaving like this. samplefile.txt: COL1|COL2|COL3|COL4
"1st Data"|"2nd ""\\\\P"" data"|"3rd data"|"4th data" This is my spark code to read data: val df = spark.read.format("csv").option("header","true").option("inferSchema","true").option("delimiter","|").load("\samplefile.xtx")
df.show(false) Some how it is combining 2 columns data into one. Spark Scala : 2.4 Version Any idea why spark is behaving like this.
... View more
Labels:
- Labels:
-
Apache Spark