question Re: What is use of Flatten in Pig? in Archives of Support Questions (Read Only)

What is use of Flatten in Pig?

rushikeshdeshmu — Sat, 12 Mar 2016 18:23:51 GMT

Hi,

Can anyone explain what is use of Flatten in Pig?

Re: What is use of Flatten in Pig?

nsabharwal — Sat, 12 Mar 2016 18:29:57 GMT

@Rushikesh Deshmukh Look at this explanation

https://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#Flatten+Operator

The FLATTEN operator looks like a UDF syntactically, but it is actually an operator that changes the structure of tuples and bags in a way that a UDF cannot. Flatten un-nests tuples as well as bags. The idea is the same, but the operation and result is different for each type of structure.

Re: What is use of Flatten in Pig?

rushikeshdeshmu — Sat, 12 Mar 2016 18:38:34 GMT

@Neeraj Sabharwal, thanks for quick reply.

Re: What is use of Flatten in Pig?

rushikeshdeshmu — Sat, 12 Mar 2016 18:47:50 GMT

I got below answer:

Sometimes there is data in a tuple or bag and if we want to remove the level of nesting from that data then Flatten modifier in Pig can be used. Flatten un-nests bags and tuples. For tuples, the Flatten operator will substitute the fields of a tuple in place of a tuple whereas un-nesting bags is a little complex because it requires creating new tuples.

Re: What is use of Flatten in Pig?

rushikeshdeshmu — Sat, 12 Mar 2016 18:54:57 GMT

@Neeraj Sabharwal, got the required answer, choosing the best answer and closing this thread.

Re: What is use of Flatten in Pig?

sangam_aruns89 — Thu, 22 Sep 2016 00:01:01 GMT

@Rushikesh Deshmukh

Flatten un-nests tuples as well as bags. consider a relation that has a tuple of the form (a, (b, c)). The expression GENERATE $0, flatten($1), will cause that tuple to become (a, b, c).

You can refer to the below link to know more and have better understanding of other operators, just in case if you need them.

https://www.qubole.com/resources/cheatsheet/pig-function-cheat-sheet/