Reply
Explorer
Posts: 7
Registered: ‎04-27-2016

Pig Script to insert new column based on filename

Hi experts,

I'm trying to add a new column to my file. I want to add the filename to each row of my file.
Filename is: 2016-06-15.txt
The schema my file is:   A           B             C
                                       7           8              13

I want to obtain:         Date          A           B             C
                             2016-06-15     7           8              13

For that I'm using Pig with following scipt:

A = LOAD 'user/cloudera/Analytics/source/file.txt' using PigStorage(' ','-tagPath');

DUMP A ;

STORE A INTO 'user/cloudera/Analytics/source/file.txt' USING PigStorage(' ');


But I'm getting an error and I don't have any log available :( Anyone can help? Many thanks!

Explorer
Posts: 7
Registered: ‎04-27-2016

Re: Pig Script to insert new column based on filename

I already found a solution. Here is the code:  


A = LOAD '/user/cloudera/Analytics/source/2013-11-01.txt' using PigStorage(' ','-tagFile');


STORE A INTO '/user/cloudera/Analytics/source/teste/2013-11-01.txt' USING PigStorage(' ');

 

;)

Highlighted
New Contributor
Posts: 1
Registered: ‎09-20-2017

Re: Pig Script to insert new column based on filename

I have a file with name YYYYMMDD_claims_portal.csv, i need only YYYYMMDD part and store this value in new column(say,date). Earlier we have 3 column like Claim,User,ID. now i need to add one more column date having value as YYYYMMDD as per file. Please help, its bit urgent.

Thanks in advance for any help you guys can provide.

 

cheers :)

Announcements