Reply
Explorer
Posts: 8
Registered: ‎02-15-2018

Error running valid Pig Script in HUE (does work within Grunt)

[ Edited ]

Hi,

 

I am running under cloudera quickstart VM CDH 5.12.0 with Pig 0.12.0. When I run the following script from Grunt shell, it works well :

pig.PNG

 

But lauched from the Hue editor it runs endlessly. It creates 2 applications :

- Pig Latin : Pig => stdout : "Invalid container ID : cloudera"

- Oozie : launcher:T => stdout :

oozie.PNG

 

From what I have read in other posts on the forum, I tried to :

1) increase the heap size of resourcemanager and nodemanager from 50 mb to 1 gb (and restarted yarn and oozie)

2) change the system user and system group in hue settings from 'hue' to 'cloudera'

 

But none resolved the issue. 

 

I don't know if this is really relevant, but here is the list of users/groups associated with hue/cloudera/oozie:

user.PNG

 

Would you please have a hint of what the problem is ?

 

Thank you in advance,

Regards, Sélim

Explorer
Posts: 8
Registered: ‎02-15-2018

Re: Error running valid Pig Script in HUE (does work within Grunt)

Looking in yarn in cloudera manager, I find my two Oozie & Pig applications :

yarnAppDetails.PNG

 

Here is an extract of the log for Oozie :

ozzie_yarnAppDetails.PNGozzie_yarnAppDetails_2.PNG

 

And for Pig :

pig_yarnAppDetails.PNGpig_yarnAppDetails_2.PNG

 

It seems to be a memory issue but I am not sure ... relaunching the job, I can see that the map task of te oozie launcher stops at 95% (100% map, 0% reduce), and the piglatin task stops at 5% (0% map, 0% reduce ...) :

yarnAppDetails_2.PNG

Please let me know if you need more info,

Regards, Sélim

Highlighted
Explorer
Posts: 8
Registered: ‎02-15-2018

Re: Error running valid Pig Script in HUE (does work within Grunt)

Last but not least, the script is properly executed if I comment the 'store' command. I have the same issue if I simply dump the variable 'month_count' instead of storing it. So the issue seems to be triggered by either a store or a dump action ...