Member since
01-22-2016
4
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 25450 | 03-10-2016 07:12 PM |
03-02-2017
09:39 AM
1 Kudo
One last thing worth mentioning. If you are using a hive action, you will need to ensure that you have the HiveServer2 URL identified on the properties tab of Hue/oozie for the action. E.g., jdbc:hive2://<HIVE_SERVER_HOSTNAME>:10000/default
... View more
03-01-2017
12:51 PM
1 Kudo
I think there was a bug in 5.5.1 where you could not review standard out and standard err outputs of shell scripts and hive scripts called from an oozie workflow. That was fixed in later versions. The job history was used as a workaround to review the error messages and find root cause. If you are calling the hive action/script from a shell script instead of a hive action you will also need to do the following. 1.Use kinit (if running a secure cluster) and kinit to an account that has access to the hive schema. 2.You must set the mapreduce.job.credentials.binary to $HADOOP_TOKEN_FILE_LOCATION in the hive script; Sample code below: HIVEQUERY='hive -S -e "SET mapreduce.job.credentials.binary=$HADOOP_TOKEN_FILE_LOCATION;set mapred.job.queue.name=myquename;SELECT count(*) FROM schema.table;"'
... View more
03-10-2016
07:12 PM
I was able to solve this by finding the URL to the Job History Server UI. Stdout and Stderr can be sen here.
... View more
03-09-2016
07:39 PM
Running CDH 5.5.1, Kerberos enabled cluster. I have a hive .sql script that runs fine interactively. When I try to launch via Oozie using Hive action it errors out. Log file below. There isn't much info in the logs. Any ideas on how to troubleshoot? Error message: Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2] Full log below: 2016-03-10 02:55:22,474 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[myhive2server.com] USER[myuser] GROUP[-] TOKEN[] APP[dev-epm-rmt-test-hive] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@:start:] Start action [0000035-160308051341496-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-03-10 02:55:22,476 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[myhive2server.com] USER[myuser] GROUP[-] TOKEN[] APP[dev-epm-rmt-test-hive] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@:start:] [***0000035-160308051341496-oozie-oozi-W@:start:***]Action status=DONE
2016-03-10 02:55:22,476 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[myhive2server.com] USER[myuser] GROUP[-] TOKEN[] APP[dev-epm-rmt-test-hive] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@:start:] [***0000035-160308051341496-oozie-oozi-W@:start:***]Action updated in DB!
2016-03-10 02:55:22,521 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[myhive2server.com] USER[myuser] GROUP[-] TOKEN[] APP[dev-epm-rmt-test-hive] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@hive2-7b7a] Start action [0000035-160308051341496-oozie-oozi-W@hive2-7b7a] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-03-10 02:55:25,848 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[myhive2server.com] USER[myuser] GROUP[-] TOKEN[] APP[dev-epm-rmt-test-hive] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@hive2-7b7a] checking action, hadoop job ID [job_1457413978008_0096] status [RUNNING]
2016-03-10 02:55:25,849 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[myhive2server.com] USER[myuser] GROUP[-] TOKEN[] APP[dev-epm-rmt-test-hive] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@hive2-7b7a] [***0000035-160308051341496-oozie-oozi-W@hive2-7b7a***]Action status=RUNNING
2016-03-10 02:55:25,849 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[myhive2server.com] USER[myuser] GROUP[-] TOKEN[] APP[dev-epm-rmt-test-hive] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@hive2-7b7a] [***0000035-160308051341496-oozie-oozi-W@hive2-7b7a***]Action updated in DB!
2016-03-10 02:55:47,945 INFO org.apache.oozie.servlet.CallbackServlet: SERVER[myhive2server.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@hive2-7b7a] callback for action [0000035-160308051341496-oozie-oozi-W@hive2-7b7a]
2016-03-10 02:55:48,076 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[myhive2server.com] USER[myuser] GROUP[-] TOKEN[] APP[dev-epm-rmt-test-hive] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@hive2-7b7a] action completed, external ID [job_1457413978008_0096]
2016-03-10 02:55:48,082 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[myhive2server.com] USER[myuser] GROUP[-] TOKEN[] APP[dev-epm-rmt-test-hive] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@hive2-7b7a] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2]
2016-03-10 02:55:48,113 INFO org.apache.oozie.command.wf.ActionEndXCommand: SERVER[myhive2server.com] USER[myuser] GROUP[-] TOKEN[] APP[dev-epm-rmt-test-hive] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@hive2-7b7a] ERROR is considered as FAILED for SLA
2016-03-10 02:55:48,137 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[myhive2server.com] USER[myuser] GROUP[-] TOKEN[] APP[dev-epm-rmt-test-hive] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@Kill] Start action [0000035-160308051341496-oozie-oozi-W@Kill] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-03-10 02:55:48,139 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[myhive2server.com] USER[myuser] GROUP[-] TOKEN[] APP[dev-epm-rmt-test-hive] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@Kill] [***0000035-160308051341496-oozie-oozi-W@Kill***]Action status=DONE
2016-03-10 02:55:48,140 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[myhive2server.com] USER[myuser] GROUP[-] TOKEN[] APP[dev-epm-rmt-test-hive] JOB[0000035-160308051341496-oozie-oozi-W] ACTION[0000035-160308051341496-oozie-oozi-W@Kill] [***0000035-160308051341496-oozie-oozi-W@Kill***]Action updated in DB! workflow.xml below: <workflow-app name="dev-epm-rmt-test-hive" xmlns="uri:oozie:workflow:0.5">
<global>
<configuration>
<property>
<name>mapreduce.job.queuename</name>
<value>root.epm</value>
</property>
</configuration>
</global>
<credentials>
<credential name="hive2" type="hive2">
<property>
<name>hive2.jdbc.url</name>
<value>jdbc:hive2://myhive2server.com:10000/default</value>
</property>
<property>
<name>hive2.server.principal</name>
<value>hive/myhive2server.com@MYREALM.ORG</value>
</property>
</credential>
</credentials>
<start to="hive2-7b7a"/>
<kill name="Kill">
<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<action name="hive2-7b7a" cred="hive2">
<hive2 xmlns="uri:oozie:hive2-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<jdbc-url>jdbc:hive2://myhive2server.com:10000/default</jdbc-url>
<password>hive2</password>
<script>/code/contacts.sql</script>
<file>/code/contacts.sql#contacts.sql</file>
</hive2>
<ok to="End"/>
<error to="Kill"/>
</action>
<end name="End"/>
</workflow-app>
... View more
Labels:
- Labels:
-
Apache Oozie