Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Pig Script to insert new column based on filename

Pig Script to insert new column based on filename

Explorer

Hi experts,

I'm trying to add a new column to my file. I want to add the filename to each row of my file.
Filename is: 2016-06-15.txt
The schema my file is:   A           B             C
                                       7           8              13

I want to obtain:         Date          A           B             C
                             2016-06-15     7           8              13

For that I'm using Pig with following scipt:

A = LOAD 'user/cloudera/Analytics/source/file.txt' using PigStorage(' ','-tagPath');

DUMP A ;

STORE A INTO 'user/cloudera/Analytics/source/file.txt' USING PigStorage(' ');


But I'm getting an error and I don't have any log available :( Anyone can help? Many thanks!

2 REPLIES 2

Re: Pig Script to insert new column based on filename

Explorer

I already found a solution. Here is the code:  


A = LOAD '/user/cloudera/Analytics/source/2013-11-01.txt' using PigStorage(' ','-tagFile');


STORE A INTO '/user/cloudera/Analytics/source/teste/2013-11-01.txt' USING PigStorage(' ');

 

;)

Re: Pig Script to insert new column based on filename

New Contributor

I have a file with name YYYYMMDD_claims_portal.csv, i need only YYYYMMDD part and store this value in new column(say,date). Earlier we have 3 column like Claim,User,ID. now i need to add one more column date having value as YYYYMMDD as per file. Please help, its bit urgent.

Thanks in advance for any help you guys can provide.

 

cheers :)