Member since
04-13-2023
3
Posts
0
Kudos Received
0
Solutions
04-18-2023
08:55 PM
@mszurap Thanks!! It worked. Upgraded to Hive2 action and things worked. However, had to add this property in hive hql : SET hive.stats.autogather=false; Guess, the flow was failing due to some issues with stats gathering. We can mark this as resolved.
... View more
04-16-2023
09:16 PM
Thanks @DianaTorres
... View more
04-13-2023
07:21 AM
Hello, I'm running Hive 3.1.3, Hue 4.10.0, Oozie 5.2.1 Through Oozie, I'm running a periodic Hive job which takes data from an HDFS path, and adds it to an existing hive table. It works fine when I'm running it using Hive shell. However, when I run it using Oozie, the underlying data in Hive gets updated correctly, but the Oozie job fails without giving any meaningful error. This is a screenshot explaining the situation. The job ran for 1hr 11m, the data in Hive got updated, but Oozie responded with a failed status. Here are the applicationLogs : Parameters:
------------------------
DATE=2023-04-12
------------------------
Hive command arguments :
--hiveconf
hive.log4j.file=/mnt/yarn/usercache/ec2-user/appcache/application_1675167116593_12260/container_1675167116593_12260_01_000001/hive-log4j.properties
--hiveconf
hive.exec.log4j.file=/mnt/yarn/usercache/ec2-user/appcache/application_1675167116593_12260/container_1675167116593_12260_01_000001/hive-exec-log4j.properties
--hiveconf
hive.log.trace.id=oozie:0005857-230202071432750-oozie-oozi-W
-f
dr_update_hive.hql
--hivevar
DATE=2023-04-12
Fetching child yarn jobs
tag id : oozie-2621f368aebc0324f3d78a262abee63e
No child applications found
=================================================================
>>> Invoking Hive command line now >>>
<<< Invocation of Hive command completed <<<
No child hadoop job is executed.
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:412)
at org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:54)
at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:225)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:219)
at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:155)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:143)
Caused by: java.lang.SecurityException: Intercepted System.exit(1)
at org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
at java.lang.Runtime.exit(Runtime.java:107)
at java.lang.System.exit(System.java:973)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:685)
at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:310)
at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:294)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:69)
... 16 more
Intercepting System.exit(1)
Failing Oozie Launcher, Main Class [org.apache.oozie.action.hadoop.HiveMain], exit code [1]
Oozie Launcher, uploading action data to HDFS sequence file: hdfs://ip-10-162-8-160.ap-south-1.compute.internal:8020/user/ec2-user/oozie-oozi/0005857-230202071432750-oozie-oozi-W/hiveScript--hive/action-data.seq
Stopping AM
Callback notification attempts left 0
Callback notification trying http://ip-10-162-8-160.ap-south-1.compute.internal:11000/oozie/callback?id=0005857-230202071432750-oozie-oozi-W@hiveScript&status=FAILED
Callback notification to http://ip-10-162-8-160.ap-south-1.compute.internal:11000/oozie/callback?id=0005857-230202071432750-oozie-oozi-W@hiveScript&status=FAILED succeeded
Callback notification succeeded I need this because the job is correct and I'm getting unnecessary FAILURE alerts. P.S. : Please tell me if any other info is needed. I'll be happy to provide
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Oozie