I have a table with 3 columns ( customer id, amount, quantity) which has 1 billion of records. My query is select cusomer id, sum(amount) from my_table group by customer_id. I wanted to understand how the mappers and reducers picks the data. I assume mappers will pick all the keys in its corresponding mappers. After that shuffler will group all the same set of keys. Then reducers will perform the sum operations and delivers it. Please correct me if im wrong here. Also When the reducers performs the sum operation will each reducers works on its key value pair which was feed from its mapper job? If the same key has huge volume which cant be handled by a single reducer can it be split into multiple reducers? ... View more

Im new to pig script to pardon me if my question is lame. I know that we can define a datatype for each atom in pig while loading it from a file. But is there a way we can define the datatype after taking a subset of it? Example: data = load 'mydata.csv' using PigStrogae(',') AS (col1:int, col2:int); subsetdata = foreach data generate col1; --> Here i need to define the col1 as int . Is there a way to feed it? ... View more

On what basis does a key value pair is generated? How to decide the no of mappers and its associated reducers? ... View more

Is there anyway to find out how long does a hive query take to get completed? ... View more

What are the differences between mr mode and tez. Why tez functions faster than mr? why few queries fail in tez but gets executed successfully in mr mode? I know the question I ask have multiple answers but I just wanted to know all the possible scenarios. Thanks ... View more