Oozie work flow - PIG action failed

[#1]

Hi I am trying to run a PIG action in oozie workflow and getting this error:

Pig Stack Trace
---------------
ERROR 2998: Unhandled internal error. org/apache/hadoop/hive/shims/ShimLoader

java.lang.NoClassDefFoundError: org/apache/hadoop/hive/shims/ShimLoader
	at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>(HiveConf.java:368)
	at org.apache.hive.hcatalog.pig.PigHCatUtil.getHCatServerUri(PigHCatUtil.java:134)
	at org.apache.hive.hcatalog.pig.HCatLoader.getSchema(HCatLoader.java:217)
	at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:175)
	at org.apache.pig.newplan.logical.relational.LOLoad.<init>(LOLoad.java:89)
	at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:901)
	at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3568)
	at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
	at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
	at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
	at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
	at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1735)
	at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1443)
	at org.apache.pig.PigServer.parseAndBuild(PigServer.java:387)
	at org.apache.pig.PigServer.executeBatch(PigServer.java:412)
	at org.apache.pig.PigServer.executeBatch(PigServer.java:398)
	at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171)
	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:234)
	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
	at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
	at org.apache.pig.Main.run(Main.java:502)
	at org.apache.pig.PigRunner.run(PigRunner.java:49)
	at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:288)
	at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:231)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
	at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.shims.ShimLoader
	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
	... 40 more
================================================================================
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2]

[#2]

@Sushil Saxena

I have a fix documented in this thread.

[#3]

Please let me know which one I need to set:

oozie.action.sharelib.for.pig=pig,hcatalog

oozie.action.sharelib.for.pig=hive,pig,hcatalog

oozie.action.sharelib.for.hive=hive,hcatalog,sqoop

Also where I need to set: In Job Config or Hive-site.xml as custom property.

[#4]

@Sushil Saxena

I'll make it easy for you, just copy this workflow from my github page it has everything you need to run on sandbox, on your own environment, copy hive-site.xml from your cluster to lib folder. To answer your question, you choose the packages depending on what you're using, if you're using hcatalog you need pig,hcatalog and hive, if you use sqoop, add sqoop, etc.

[#5]

It works fine when we made changes in workflow.xml for oozie 4.2. But when we are running Falcon process and then system will create the oozie workflow. At PIG action, it got failed with ShimsLoader error. Actually falcon is not picking changes made in oozie-site.xml:

oozie.action.sharelib.for.pig=pig,hcatalog,hive

I also tried restarting Falcon, OOzie, Hive bit still the not picking up the above property.

[#6]

@Sushil Saxena

right click pig and refresh client configs. As far as Falcon not picking up Oozie configs, have you confirmed that there are no stale services in existence?

[#7 · Accepted solution]

Sushil Saxena: Regarding issue in Falcon there is FALCON-1787 jira created.

If you are running Falcon process, updating "oozie.action.sharelib.for.pig" in Oozie-site.xml will not help. Falcon generates the pig action at the run time & generates the workflow, it uses pig-action.xml defined in Falcon codebase. pig-action.xml that Falcon uses does not have hive in the share lib config "oozie.action.sharelib.for.pig".

Workflow action configuration generated by Falcon is overriding the one defined in ooze-site.xml.

Few work arounds:

1> In the process entity definition add new custom property. Resubmit the process.

<properties> <property name="oozie.action.sharelib.for.pig" value="hive,pig,hcatalog"/> </properties>

2> Update pig-action.xml to have hive in the share lib config and repackage falcon-oozie-adaptor-<version>.jar and replace jar at "/usr/hdp/current/falcon-server/webapp/falcon/WEB-INF/lib” and restart Falcon

3> If you have Falcon code downloaded then update pig-action.xml at oozie/src/main/resources/action/process/pig-action.xml and then build falcon and reinstall it