What is use of Flatten in Pig?

[#1]

Hi,

Can anyone explain what is use of Flatten in Pig?

[#2 · Accepted solution]

@Rushikesh Deshmukh Look at this explanation

https://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#Flatten+Operator

The FLATTEN operator looks like a UDF syntactically, but it is actually an operator that changes the structure of tuples and bags in a way that a UDF cannot. Flatten un-nests tuples as well as bags. The idea is the same, but the operation and result is different for each type of structure.

[#3]

@Neeraj Sabharwal, thanks for quick reply.

[#4]

@Neeraj Sabharwal, got the required answer, choosing the best answer and closing this thread.

[#5]

I got below answer:

Sometimes there is data in a tuple or bag and if we want to remove the level of nesting from that data then Flatten modifier in Pig can be used. Flatten un-nests bags and tuples. For tuples, the Flatten operator will substitute the fields of a tuple in place of a tuple whereas un-nesting bags is a little complex because it requires creating new tuples.

[#6]

@Rushikesh Deshmukh

Flatten un-nests tuples as well as bags. consider a relation that has a tuple of the form (a, (b, c)). The expression GENERATE $0, flatten($1), will cause that tuple to become (a, b, c).

You can refer to the below link to know more and have better understanding of other operators, just in case if you need them.

https://www.qubole.com/resources/cheatsheet/pig-function-cheat-sheet/