Created 03-12-2016 10:23 AM
Hi,
Can anyone explain what is use of Flatten in Pig?
Created 03-12-2016 10:29 AM
@Rushikesh Deshmukh Look at this explanation
https://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#Flatten+Operator
The FLATTEN operator looks like a UDF syntactically, but it is actually an operator that changes the structure of tuples and bags in a way that a UDF cannot. Flatten un-nests tuples as well as bags. The idea is the same, but the operation and result is different for each type of structure.
Created 03-12-2016 10:29 AM
@Rushikesh Deshmukh Look at this explanation
https://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#Flatten+Operator
The FLATTEN operator looks like a UDF syntactically, but it is actually an operator that changes the structure of tuples and bags in a way that a UDF cannot. Flatten un-nests tuples as well as bags. The idea is the same, but the operation and result is different for each type of structure.
Created 03-12-2016 10:38 AM
@Neeraj Sabharwal, thanks for quick reply.
Created 03-12-2016 10:54 AM
@Neeraj Sabharwal, got the required answer, choosing the best answer and closing this thread.
Created 03-12-2016 10:47 AM
I got below answer:
Sometimes there is data in a tuple or bag and if we want to remove the level of nesting from that data then Flatten modifier in Pig can be used. Flatten un-nests bags and tuples. For tuples, the Flatten operator will substitute the fields of a tuple in place of a tuple whereas un-nesting bags is a little complex because it requires creating new tuples.
Created 09-21-2016 05:01 PM
Flatten un-nests tuples as well as bags. consider a relation that has a tuple of the form (a, (b, c)). The expression GENERATE $0, flatten($1), will cause that tuple to become (a, b, c).
You can refer to the below link to know more and have better understanding of other operators, just in case if you need them.
https://www.qubole.com/resources/cheatsheet/pig-function-cheat-sheet/