Support Questions

Find answers, ask questions, and share your expertise

Scala Flatten multi valued column into rows

avatar
Expert Contributor

I am working with scala and i have a dataframe with one of its columns containing several values delimited by a comma. How can i turn these rows

["1", "x,y,z,", "A"]

["2", "x,y", "B"]

into

["1", "x,", "A"]

["1", "y,", "A"]

["1", "z", "A"]

["2", "x", "B"]

["2", "y", "B"]

1 ACCEPTED SOLUTION

avatar
Expert Contributor

this is the code i came up with, is there a better approach?

val ds = filteredDF.as[(Integer, String, String, String, String, Double, Integer)]
var df = ds.flatMap { 
  case (x1, x2, x3, x4, x5, x6, x7) => x3.split(",").map((x1, x2, _, x4, x5, x6, x7))
}.toDF

View solution in original post

1 REPLY 1

avatar
Expert Contributor

this is the code i came up with, is there a better approach?

val ds = filteredDF.as[(Integer, String, String, String, String, Double, Integer)]
var df = ds.flatMap { 
  case (x1, x2, x3, x4, x5, x6, x7) => x3.split(",").map((x1, x2, _, x4, x5, x6, x7))
}.toDF