Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Solved
Go to solution
Scala Flatten multi valued column into rows
Labels:
- Labels:
-
Apache Spark
Expert Contributor
Created 07-26-2017 06:33 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am working with scala and i have a dataframe with one of its columns containing several values delimited by a comma. How can i turn these rows
["1", "x,y,z,", "A"]
["2", "x,y", "B"]
into
["1", "x,", "A"]
["1", "y,", "A"]
["1", "z", "A"]
["2", "x", "B"]
["2", "y", "B"]
1 ACCEPTED SOLUTION
Expert Contributor
Created 07-26-2017 09:57 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
this is the code i came up with, is there a better approach?
val ds = filteredDF.as[(Integer, String, String, String, String, Double, Integer)] var df = ds.flatMap { case (x1, x2, x3, x4, x5, x6, x7) => x3.split(",").map((x1, x2, _, x4, x5, x6, x7)) }.toDF
1 REPLY 1
Expert Contributor
Created 07-26-2017 09:57 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
this is the code i came up with, is there a better approach?
val ds = filteredDF.as[(Integer, String, String, String, String, Double, Integer)] var df = ds.flatMap { case (x1, x2, x3, x4, x5, x6, x7) => x3.split(",").map((x1, x2, _, x4, x5, x6, x7)) }.toDF
