Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Error running valid Pig Script in HUE (does work within Grunt)

Error running valid Pig Script in HUE (does work within Grunt)

Explorer

Hi,

 

I am running under cloudera quickstart VM CDH 5.12.0 with Pig 0.12.0. When I run the following script from Grunt shell, it works well :

pig.PNG

 

But lauched from the Hue editor it runs endlessly. It creates 2 applications :

- Pig Latin : Pig => stdout : "Invalid container ID : cloudera"

- Oozie : launcher:T => stdout :

oozie.PNG

 

From what I have read in other posts on the forum, I tried to :

1) increase the heap size of resourcemanager and nodemanager from 50 mb to 1 gb (and restarted yarn and oozie)

2) change the system user and system group in hue settings from 'hue' to 'cloudera'

 

But none resolved the issue. 

 

I don't know if this is really relevant, but here is the list of users/groups associated with hue/cloudera/oozie:

user.PNG

 

Would you please have a hint of what the problem is ?

 

Thank you in advance,

Regards, Sélim

2 REPLIES 2

Re: Error running valid Pig Script in HUE (does work within Grunt)

Explorer

Looking in yarn in cloudera manager, I find my two Oozie & Pig applications :

yarnAppDetails.PNG

 

Here is an extract of the log for Oozie :

ozzie_yarnAppDetails.PNGozzie_yarnAppDetails_2.PNG

 

And for Pig :

pig_yarnAppDetails.PNGpig_yarnAppDetails_2.PNG

 

It seems to be a memory issue but I am not sure ... relaunching the job, I can see that the map task of te oozie launcher stops at 95% (100% map, 0% reduce), and the piglatin task stops at 5% (0% map, 0% reduce ...) :

yarnAppDetails_2.PNG

Please let me know if you need more info,

Regards, Sélim

Highlighted

Re: Error running valid Pig Script in HUE (does work within Grunt)

Explorer

Last but not least, the script is properly executed if I comment the 'store' command. I have the same issue if I simply dump the variable 'month_count' instead of storing it. So the issue seems to be triggered by either a store or a dump action ...