09-28-2015 12:36 AM
My question seems to be unclear. I will explain with an example.
My task is to append the first row of my file to all the remaining rows(millions of rows) in that file.
I have an input file of 200 MB as below :
zsd 456 def 123123 adadfdf
jdf 342 dsf 234234 asdfasdf
tkj 745 lkh 413531 kljkdfga
Output Should be as :
zsd 456 def 123123 adadfdf abc123
jdf 342 dsf 234234 sdfasdf abc123
tkj 745 lkh 435345 kljkdfga abc123
What I did was, stored the first row in a member variable of map class and then appended to all the remaining rows.
But the problem is, as the 200 MB file is splitted in 4 blocks (3*64 MB and 1*8 MB) the ouptut from the first mapper resulted correctly,
but the output of other mappers are appending the first row of their respective blocks.
How could I complete this task, I need to append the same "abc123" to all the rows of my input file.
PS: My code contains only mapper and no reducer.
09-28-2015 07:31 AM
09-28-2015 11:48 PM - edited 09-28-2015 11:51 PM
Thankyou, your suggestion might solve the current problem.
I have similar issue where in there are 4 headers in my file with corresponding records. These headers have to be appended to their respective records.
Now how can I achieve that ? Any suggestions ?