03-01-2018 06:20 AM - last edited on 03-01-2018 08:01 AM by cjervis
I am running under cloudera quickstart VM CDH 5.12.0 with Pig 0.12.0. When I run the following script from Grunt shell, it works well :
But lauched from the Hue editor it runs endlessly. It creates 2 applications :
- Pig Latin : Pig => stdout : "Invalid container ID : cloudera"
- Oozie : launcher:T => stdout :
From what I have read in other posts on the forum, I tried to :
1) increase the heap size of resourcemanager and nodemanager from 50 mb to 1 gb (and restarted yarn and oozie)
2) change the system user and system group in hue settings from 'hue' to 'cloudera'
But none resolved the issue.
I don't know if this is really relevant, but here is the list of users/groups associated with hue/cloudera/oozie:
Would you please have a hint of what the problem is ?
Thank you in advance,
03-01-2018 07:03 AM
Looking in yarn in cloudera manager, I find my two Oozie & Pig applications :
Here is an extract of the log for Oozie :
And for Pig :
It seems to be a memory issue but I am not sure ... relaunching the job, I can see that the map task of te oozie launcher stops at 95% (100% map, 0% reduce), and the piglatin task stops at 5% (0% map, 0% reduce ...) :
Please let me know if you need more info,
03-01-2018 07:47 AM
Last but not least, the script is properly executed if I comment the 'store' command. I have the same issue if I simply dump the variable 'month_count' instead of storing it. So the issue seems to be triggered by either a store or a dump action ...