Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

What is use of Flatten in Pig?

SOLVED Go to solution
Highlighted

What is use of Flatten in Pig?

Hi,

Can anyone explain what is use of Flatten in Pig?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: What is use of Flatten in Pig?

@Rushikesh Deshmukh Look at this explanation

https://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#Flatten+Operator

The FLATTEN operator looks like a UDF syntactically, but it is actually an operator that changes the structure of tuples and bags in a way that a UDF cannot. Flatten un-nests tuples as well as bags. The idea is the same, but the operation and result is different for each type of structure.

5 REPLIES 5

Re: What is use of Flatten in Pig?

@Rushikesh Deshmukh Look at this explanation

https://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#Flatten+Operator

The FLATTEN operator looks like a UDF syntactically, but it is actually an operator that changes the structure of tuples and bags in a way that a UDF cannot. Flatten un-nests tuples as well as bags. The idea is the same, but the operation and result is different for each type of structure.

Re: What is use of Flatten in Pig?

@Neeraj Sabharwal, thanks for quick reply.

Re: What is use of Flatten in Pig?

@Neeraj Sabharwal, got the required answer, choosing the best answer and closing this thread.

Re: What is use of Flatten in Pig?

I got below answer:

Sometimes there is data in a tuple or bag and if we want to remove the level of nesting from that data then Flatten modifier in Pig can be used. Flatten un-nests bags and tuples. For tuples, the Flatten operator will substitute the fields of a tuple in place of a tuple whereas un-nesting bags is a little complex because it requires creating new tuples.

Re: What is use of Flatten in Pig?

New Contributor

@Rushikesh Deshmukh

Flatten un-nests tuples as well as bags. consider a relation that has a tuple of the form (a, (b, c)). The expression GENERATE $0, flatten($1), will cause that tuple to become (a, b, c).

You can refer to the below link to know more and have better understanding of other operators, just in case if you need them.

https://www.qubole.com/resources/cheatsheet/pig-function-cheat-sheet/