question Re: How to get EPOCH time in PIG? in Archives of Support Questions (Read Only)

How to get EPOCH time in PIG?

mohan221213 — Fri, 02 Sep 2016 16:14:47 GMT

I am completely new to pig. Started to learn on my own and trying to know PIG. Can anyone please suggest me to get the current system epoch time using pig.

I need the output as

epochtime_1234567890

And how can we call the shell script in PIG script

Because I have already written to get EPOCH time in Shell script.

Any Help.

Thanks in Advance.

Mohan.v

Re: How to get EPOCH time in PIG?

matt_andruff — Fri, 02 Sep 2016 20:58:58 GMT

Hey Mohan,

It might be cleaner to just write a UDF. I certainly feel like PIG has purposely left this functionality out to encourage the use of datetime over the use of epochs.

Here's one way to achieve what you want to do:

a = load 'test.data';
b = foreach a GENERATE *, CONCAT( 'epochtime_', (chararray)MilliSecondsBetween(CurrentTime(),ToDate(0)))
;dump b;

Hope this helps.

Re: How to get EPOCH time in PIG?

mohan221213 — Fri, 02 Sep 2016 21:09:46 GMT

Thank for your reply Matt. I think i might got the solution

b =foreach a GENERATE CONCAT('epochtime_',(chararray) ToUnixTime(CurrentTime(0));

and i am able to get the output as i want.

but can you suggest me for,

reading a words.txt file having words and get the output as

word1_epochtime word2_epochtime.

Please don't hesitate to reply.

Re: How to get EPOCH time in PIG?

mohan221213 — Fri, 02 Sep 2016 21:21:19 GMT

I think i got it too.

please correct me if im wrong

A = LOAD 'words.txt' AS (word:chararray);

B = FOREACH A GENERATE CONCAT(CONCAT(A.word,'_'),(chararray)ToUnixTime(CurrentTime());


dump B;

Re: How to get EPOCH time in PIG?

matt_andruff — Sat, 03 Sep 2016 02:50:52 GMT

That Looks better than what I proposed. Thumbs up!