question Pig script fails to write output on first attempt in Archives of Support Questions (Read Only)

Pig script fails to write output on first attempt

IT.Services — Tue, 21 Apr 2026 14:00:00 GMT

My pig script (running through Hue) fails to store the results into HDFS on the first attempt. Immediately after attempting to store the data the entire pig script restarts. The script will then complete successfully on the second attempt. Here is my pig script:

offers = LOAD '/tmp/file.txt' USING PigStorage AS (tabid:CHARARRAY, offerNum:CHARARRAY);

describe offers;
offers5= LIMIT offers 5;
dump offers5;

STORE offers INTO '/tmp/folder' USING PigStorage();

I think my pig script is written poorly, can you identify why the entire script would restart? I can't find anything useful in the logs! Where can I look to try to resolve this issue?

Re: Pig script fails to write output on first attempt

IT.Services — Wed, 12 Nov 2014 19:09:52 GMT

I stumbled upon articles written about how to use STORE and DUMP appropriately in a pig script. It seems that I have been using a DUMP and a STORE command in our scripts to output some debugging information. Instead I should only be using the STORE command in our scripts. DUMP is used only for debugging. If you combine the two commands the script will run TWICE!

From Apache (http://pig.apache.org/docs/r0.12.0/perf.html#store-dump):

Store vs. Dump

With multi-query exection, you want to use STORE to save (persist) your results. You do not want to use DUMP as it will disable multi-query execution and is likely to slow down execution. (If you have included DUMP statements in your scripts for debugging purposes, you should remove them.)

DUMP Example: In this script, because the DUMP command is interactive, the multi-query execution will be disabled and two separate jobs will be created to execute this script. The first job will execute A > B > DUMP while the second job will execute A > B > C > STORE.

A = LOAD 'input' AS (x, y, z);
B = FILTER A BY x > 5;
DUMP B;
C = FOREACH B GENERATE y, z;
STORE C INTO 'output';

STORE Example: In this script, multi-query optimization will kick in allowing the entire script to be executed as a single job. Two outputs are produced: output1 and output2.

A = LOAD 'input' AS (x, y, z);
B = FILTER A BY x > 5;
STORE B INTO 'output1';
C = FOREACH B GENERATE y, z;
STORE C INTO 'output2';

how to split tuple into to different columns in pig

HadoopHelp — Wed, 24 Jul 2019 10:59:42 GMT

Dear All .

Below is my data format in pig output file but $1 column contains name :-

name dob gender address

(LACTE^MIRAM,19659802,F,4395 NOREY AVE^^ATTIC CITY^N9J^09901)
(JSON^MAW,19820309,M,COUNTRY CLUB DR^^NORELD^NJ^99225)

but want like below:-

firstname lastname dob gender address

(LACTE,MIRAM,19659802,F,4395 NOREY AVE^^ATTIC CITY^N9J^09901)
(JSON,MAW,19820309,M,COUNTRY CLUB DR^^NORELD^NJ^99225)

Thanks

HadoopHelp