Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Scala Flatten multi valued column into rows

Solved Go to solution

Scala Flatten multi valued column into rows

Expert Contributor

I am working with scala and i have a dataframe with one of its columns containing several values delimited by a comma. How can i turn these rows

["1", "x,y,z,", "A"]

["2", "x,y", "B"]

into

["1", "x,", "A"]

["1", "y,", "A"]

["1", "z", "A"]

["2", "x", "B"]

["2", "y", "B"]

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Scala Flatten multi valued column into rows

Expert Contributor

this is the code i came up with, is there a better approach?

val ds = filteredDF.as[(Integer, String, String, String, String, Double, Integer)]
var df = ds.flatMap { 
  case (x1, x2, x3, x4, x5, x6, x7) => x3.split(",").map((x1, x2, _, x4, x5, x6, x7))
}.toDF

View solution in original post

1 REPLY 1
Highlighted

Re: Scala Flatten multi valued column into rows

Expert Contributor

this is the code i came up with, is there a better approach?

val ds = filteredDF.as[(Integer, String, String, String, String, Double, Integer)]
var df = ds.flatMap { 
  case (x1, x2, x3, x4, x5, x6, x7) => x3.split(",").map((x1, x2, _, x4, x5, x6, x7))
}.toDF

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here