question Re: Scala Flatten multi valued column into rows in Archives of Support Questions (Read Only)

Scala Flatten multi valued column into rows

afdebbas — Thu, 27 Jul 2017 01:33:07 GMT

I am working with scala and i have a dataframe with one of its columns containing several values delimited by a comma. How can i turn these rows

["1", "x,y,z,", "A"]

["2", "x,y", "B"]

into

["1", "x,", "A"]

["1", "y,", "A"]

["1", "z", "A"]

["2", "x", "B"]

["2", "y", "B"]

Re: Scala Flatten multi valued column into rows

afdebbas — Thu, 27 Jul 2017 04:57:57 GMT

this is the code i came up with, is there a better approach?

val ds = filteredDF.as[(Integer, String, String, String, String, Double, Integer)]
var df = ds.flatMap { 
  case (x1, x2, x3, x4, x5, x6, x7) => x3.split(",").map((x1, x2, _, x4, x5, x6, x7))
}.toDF