Support Questions

Find answers, ask questions, and share your expertise
Announcements
Welcome to the upgraded Community! Read this blog to see What’s New!

Scala Flatten multi valued column into rows

avatar
Expert Contributor

I am working with scala and i have a dataframe with one of its columns containing several values delimited by a comma. How can i turn these rows

["1", "x,y,z,", "A"]

["2", "x,y", "B"]

into

["1", "x,", "A"]

["1", "y,", "A"]

["1", "z", "A"]

["2", "x", "B"]

["2", "y", "B"]

1 ACCEPTED SOLUTION

avatar
Expert Contributor

this is the code i came up with, is there a better approach?

val ds = filteredDF.as[(Integer, String, String, String, String, Double, Integer)]
var df = ds.flatMap { 
  case (x1, x2, x3, x4, x5, x6, x7) => x3.split(",").map((x1, x2, _, x4, x5, x6, x7))
}.toDF

View solution in original post

1 REPLY 1

avatar
Expert Contributor

this is the code i came up with, is there a better approach?

val ds = filteredDF.as[(Integer, String, String, String, String, Double, Integer)]
var df = ds.flatMap { 
  case (x1, x2, x3, x4, x5, x6, x7) => x3.split(",").map((x1, x2, _, x4, x5, x6, x7))
}.toDF
Labels