Reply
Highlighted
New Contributor
Posts: 2
Registered: ‎12-20-2017

Issue with use of STORE clause in HUE/PIG

Hi,

 

I am currently running VM 5.12.0.0.

 

When I run a Pig script (using HUE/PIG) and include a STORE clause the script never completes.

 

I have a small file being processed just to make sure it isn't data related.

 

The script is as follows:

 

mytags = LOAD 'stackexchange/tags-no-header.csv' USING PigStorage(',') as (Id,TagName,CountTags:int,ExcerptPostId,WikiPostId);

 

thetags = FOREACH mytags GENERATE Id,TagName,CountTags;

 

orderedtags = ORDER thetags BY CountTags DESC;

 

ILLUSTRATE mytags;

 

STORE orderedtags INTO 'stackexchange/outputtags';

 

 

 

If I leave the last line (STORE clause) out the script works fine in seconds.  Once I place the STORE clause in I am seeing this in the logs:

 

2017-12-20 06:32:48,580 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation  - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2017-12-20 06:32:48,669 [main] INFO  org.apache.pig.data.SchemaTupleBackend  - Key [pig.schematuple] was not set... will not generate code.
(539,snippets,13,7987,7986)
-----------------------------------------------------------------------------------------------------------------------------
| mytags     | Id:bytearray   | TagName:bytearray   | CountTags:int   | ExcerptPostId:bytearray   | WikiPostId:bytearray    | 
-----------------------------------------------------------------------------------------------------------------------------
|            | 539            | snippets            | 13              | 7987                      | 7986                    | 
-----------------------------------------------------------------------------------------------------------------------------

Heart beat
Heart beat

It just continues to print out Heart beat.

 

The directory I am trying to write to does not exist (otherwise the job gets killed).

 

So I am a little lost now on this one.  It feels like there is a connection issue.  Nothing obvious is showing in the logs.  I have restarted all services and tried again, just in case that is the issue.

 

Any ideas?

 

Thanks.

 

 

Announcements