Member since
07-06-2015
3
Posts
0
Kudos Received
0
Solutions
07-07-2015
08:27 AM
Thank you for looking into the issue. When I run the job is KILLED aftersometime by oozie. In the log ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2] 2015-07-06 11:08:25,314 WARN PigActionExecutor:523 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@pig-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2] 2015-07-06 11:08:25,528 INFO ActionEndXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@pig-node] ERROR is considered as FAILED for SLA 2015-07-06 11:08:25,644 INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@fail] Start action [0000001-150706100519676-oozie-oozi-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
... View more
07-06-2015
11:12 AM
resolved above issue by adding sharelib path in job.properties file but the endup with other issue. 2015-07-06 11:07:20,962 INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@:start:] Start action [0000001-150706100519676-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2015-07-06 11:07:20,962 INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@:start:] [***0000001-150706100519676-oozie-oozi-W@:start:***]Action status=DONE 2015-07-06 11:07:20,963 INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@:start:] [***0000001-150706100519676-oozie-oozi-W@:start:***]Action updated in DB! 2015-07-06 11:07:21,105 INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@pig-node] Start action [0000001-150706100519676-oozie-oozi-W@pig-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2015-07-06 11:07:28,024 INFO PigActionExecutor:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@pig-node] checking action, hadoop job ID [job_1436201602998_0002] status [RUNNING] 2015-07-06 11:07:28,028 INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@pig-node] [***0000001-150706100519676-oozie-oozi-W@pig-node***]Action status=RUNNING 2015-07-06 11:07:28,028 INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@pig-node] [***0000001-150706100519676-oozie-oozi-W@pig-node***]Action updated in DB! 2015-07-06 11:08:24,517 INFO CallbackServlet:520 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@pig-node] callback for action [0000001-150706100519676-oozie-oozi-W@pig-node] 2015-07-06 11:08:25,284 INFO PigActionExecutor:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@pig-node] Hadoop Jobs launched : [job_1436201602998_0003] 2015-07-06 11:08:25,296 INFO PigActionExecutor:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@pig-node] action completed, external ID [job_1436201602998_0002] 2015-07-06 11:08:25,314 WARN PigActionExecutor:523 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@pig-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2] 2015-07-06 11:08:25,528 INFO ActionEndXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@pig-node] ERROR is considered as FAILED for SLA 2015-07-06 11:08:25,644 INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@fail] Start action [0000001-150706100519676-oozie-oozi-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2015-07-06 11:08:25,646 INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@fail] [***0000001-150706100519676-oozie-oozi-W@fail***]Action status=DONE 2015-07-06 11:08:25,646 INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000001-150706100519676-oozie-oozi-W] ACTION[0000001-150706100519676-oozie-oozi-W@fail] [***0000001-150706100519676-oozie-oozi-W@fail***]Action updated in DB!
... View more
07-06-2015
10:17 AM
Pig and Oozie job.properties nameNode=hdfs://localhost:8020 jobTracker=localhost:8032 queueName=default oozie.libpath=/oozie/lib oozie.wf.application.path=${nameNode}/oozie workflow.xml <workflow-app name='pig-wf' xmlns="uri:oozie:workflow:0.3"> <start to='pig-node'/> <action name='pig-node'> <pig> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <prepare> <delete path="${nameNode}/oozie/pig/out"/></prepare> <configuration> <property> <name>mapred.job.queue.name</name> <value>${queueName}</value> </property> </configuration> <script>script.pig</script> </pig> <ok to="end"/> <error to="fail"/> </action> <kill name="fail"> <message>Pig failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app> script.pig A = LOAD 'data.csv'; store A into 'oozie_out'; Folder Structure: [cloudera@quickstart oozie]$ hdfs dfs -ls /oozie Found 5 items -rw-r--r-- 1 oozie supergroup 29 2015-07-05 10:55 /oozie/data.csv drwxr-xr-x - oozie supergroup 0 2015-07-05 10:55 /oozie/lib -rw-r--r-- 1 oozie supergroup 148 2015-07-05 10:55 /oozie/pig.properties -rw-r--r-- 1 oozie supergroup 48 2015-07-05 10:55 /oozie/script.pig -rw-r--r-- 1 oozie supergroup 848 2015-07-05 10:55 /oozie/workflow.xml libfolder [cloudera@quickstart oozie]$ hdfs dfs -ls /oozie/lib Found 7 items -rw-r--r-- 1 oozie supergroup 338540 2015-07-05 10:55 /oozie/lib/datafu-1.1.0-cdh5.4.0.jar -rw-r--r-- 1 oozie supergroup 338540 2015-07-05 10:55 /oozie/lib/datafu.jar -rw-r--r-- 1 oozie supergroup 9741 2015-07-05 10:55 /oozie/lib/pig-0.12.0-cdh5.4.0-smoketests.jar -rw-r--r-- 1 oozie supergroup 8377412 2015-07-05 10:55 /oozie/lib/pig-0.12.0-cdh5.4.0-withouthadoop.jar -rw-r--r-- 1 oozie supergroup 24380084 2015-07-05 10:55 /oozie/lib/pig-0.12.0-cdh5.4.0.jar -rw-r--r-- 1 oozie supergroup 8377412 2015-07-05 10:55 /oozie/lib/pig.jar -rw-r--r-- 1 oozie supergroup 378081 2015-07-05 10:55 /oozie/lib/piggybank.jar Cloudera VM : 5.4.0 Extra Info : Oozie works fine with MR Jobs. Can you please help me to resolve the issue? Thanks, Sree
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Oozie
-
Apache Pig
-
HDFS