Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 03-06-2017 09:05 PM
I am trying to run oozie workflow from Hue. I am trying to run a hive job in the workflow and that seem to be working but overall I am getting error in the oozie workflow.
Error:
Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [10096]
I have a jdbc driver in hdfs and
HDP 2.5.3
Hue version: 2.6.1-37
Log:
2017-03-06 15:47:46,557 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@:start:] Start action [0000005-170304185753616-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2017-03-06 15:47:46,557 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@:start:] [***0000005-170304185753616-oozie-oozi-W@:start:***]Action status=DONE 2017-03-06 15:47:46,557 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@:start:] [***0000005-170304185753616-oozie-oozi-W@:start:***]Action updated in DB! 2017-03-06 15:47:46,611 INFO WorkflowNotificationXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000005-170304185753616-oozie-oozi-W@:start: 2017-03-06 15:47:46,611 INFO WorkflowNotificationXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000005-170304185753616-oozie-oozi-W 2017-03-06 15:47:46,627 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] Start action [0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2017-03-06 15:47:48,278 INFO HiveActionExecutor:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] checking action, hadoop job ID [job_1488578860595_0014] status [RUNNING] 2017-03-06 15:47:48,281 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] [***0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean***]Action status=RUNNING 2017-03-06 15:47:48,281 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] [***0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean***]Action updated in DB! 2017-03-06 15:47:48,287 INFO WorkflowNotificationXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] No Notification URL is defined. Therefore nothing to notify for job 0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean 2017-03-06 15:48:51,019 INFO CallbackServlet:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] callback for action [0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] 2017-03-06 15:48:51,151 INFO HiveActionExecutor:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] Trying to get job [job_1488578860595_0014], attempt [1] 2017-03-06 15:48:51,239 INFO HiveActionExecutor:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] Hadoop Jobs launched : [job_1488578860595_0015] 2017-03-06 15:48:51,241 INFO HiveActionExecutor:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] action completed, external ID [job_1488578860595_0014] 2017-03-06 15:48:51,255 WARN HiveActionExecutor:523 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [10096] 2017-03-06 15:48:51,281 INFO ActionEndXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] ERROR is considered as FAILED for SLA 2017-03-06 15:48:51,320 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@kill] Start action [0000005-170304185753616-oozie-oozi-W@kill] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2017-03-06 15:48:51,320 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@kill] [***0000005-170304185753616-oozie-oozi-W@kill***]Action status=DONE 2017-03-06 15:48:51,320 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@kill] [***0000005-170304185753616-oozie-oozi-W@kill***]Action updated in DB! 2017-03-06 15:48:51,373 INFO WorkflowNotificationXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@kill] No Notification URL is defined. Therefore nothing to notify for job 0000005-170304185753616-oozie-oozi-W@kill 2017-03-06 15:48:51,374 INFO WorkflowNotificationXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000005-170304185753616-oozie-oozi-W 2017-03-06 15:48:51,374 INFO WorkflowNotificationXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] No Notification URL is defined. Therefore nothing to notify for job 0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean
workflow.xml
<workflow-app name="test" xmlns="uri:oozie:workflow:0.4">
<start to="daily_snapshot_clean"/>
<action name="daily_snapshot_clean">
<hive xmlns="uri:oozie:hive-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<script>/user/eedc/oozie/sbi/hive/daily/daily_snapshot_clean_sbi_master_start.hql</script>
</hive>
<ok to="end"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
Created 03-07-2017 12:51 AM
@Prabhat Ratnala The error is coming from Hive rather than oozie.
Error 10096 is a SemanticException: Dynamic partition strict mode requires at least one static partition column. To turn this off set hive.exec.dynamic.partition.mode=nonstrict
You can verify this by running your hive script without oozie. Once that work, then try running with oozie.
Created 03-07-2017 12:51 AM
@Prabhat Ratnala The error is coming from Hive rather than oozie.
Error 10096 is a SemanticException: Dynamic partition strict mode requires at least one static partition column. To turn this off set hive.exec.dynamic.partition.mode=nonstrict
You can verify this by running your hive script without oozie. Once that work, then try running with oozie.
Created 03-07-2017 07:07 PM
You are right. It was error with hive. Setting hive.exec.dynamic.partition.mode=nonstrict resolved the issue. Thank you
Created on 12-27-2023 02:32 AM - edited 12-27-2023 02:52 AM
Hi All
I am new to the CDH and the similar error I am also getting when run Oozie job through Java client application. The error Logs displays like below
Main error is " No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W@spark-node"
USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[] No results found
2023-12-22 03:33:19,121 INFO THREAD[qtp236002428-766348] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] Start action [0000040-231010100945447-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2023-12-22 03:33:19,125 INFO THREAD[qtp236002428-766348] org.apache.oozie.action.control.StartActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] Starting action
2023-12-22 03:33:19,130 INFO THREAD[qtp236002428-766348] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] [***0000040-231010100945447-oozie-oozi-W@:start:***]Action status=DONE
2023-12-22 03:33:19,131 INFO THREAD[qtp236002428-766348] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] [***0000040-231010100945447-oozie-oozi-W@:start:***]Action updated in DB!
2023-12-22 03:33:19,146 INFO THREAD[qtp236002428-766348] org.apache.oozie.action.control.StartActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] Action ended with external status [OK]
2023-12-22 03:33:19,173 INFO THREAD[qtp236002428-766348] org.apache.oozie.service.JPAService: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] No results found
2023-12-22 03:33:19,186 INFO THREAD[CallableQueue-7] org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[ln000xhstm0002.federated.fds] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W@:start:
2023-12-22 03:33:19,186 INFO THREAD[CallableQueue-37] org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[ln000xhstm0002.federated.fds] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W
2023-12-22 03:33:19,234 INFO THREAD[CallableQueue-7] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Start action [0000040-231010100945447-oozie-oozi-W@spark-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2023-12-22 03:33:19,238 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Starting action. Getting Action File System
2023-12-22 03:33:19,249 INFO THREAD[CallableQueue-7] org.apache.oozie.service.SparkConfigurationService: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] The atlas-application.properties does not exists for *: /etc/spark/conf/atlas-application.properties
2023-12-22 03:33:19,256 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Initializing sharelib excluder for action if specified.
2023-12-22 03:33:19,259 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.ShareLibExcluder: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Sharelib exclude pattern not configured, skipping.
2023-12-22 03:33:19,266 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Initializing sharelib excluder for action if specified.
2023-12-22 03:33:19,266 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.ShareLibExcluder: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Sharelib exclude pattern not configured, skipping.
2023-12-22 03:33:19,282 WARN THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] No credential properties found for action : 0000040-231010100945447-oozie-oozi-W@spark-node, cred : null
2023-12-22 03:33:19,287 WARN THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Invalid configuration value [null] defined for launcher max attempts count, using default [2].
2023-12-22 03:33:19,287 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.YarnACLHandler: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Not setting ACLs because mapreduce.cluster.acls.enabled is set to false
2023-12-22 03:33:19,652 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] checking action, hadoop job ID [application_1696946942137_0018] status [RUNNING]
2023-12-22 03:33:19,654 INFO THREAD[CallableQueue-7] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] [***0000040-231010100945447-oozie-oozi-W@spark-node***]Action status=RUNNING
2023-12-22 03:33:19,654 INFO THREAD[CallableQueue-7] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] [***0000040-231010100945447-oozie-oozi-W@spark-node***]Action updated in DB!
2023-12-22 03:33:19,666 INFO THREAD[CallableQueue-4] org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[ln000xhstm0002.federated.fds] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W@spark-node
2023-12-22 03:43:40,218 INFO THREAD[CallableQueue-6] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] action completed, external ID [application_1696946942137_0018]
2023-12-22 03:43:40,218 WARN THREAD[CallableQueue-6] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Launcher AM died, check Hadoop LOG for job [ln000xhstm0002.federated.fds:8032:application_1696946942137_0018]
2023-12-22 03:43:40,235 INFO THREAD[CallableQueue-6] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Action ended with external status [FAILED/KILLED]
2023-12-22 03:43:40,241 INFO THREAD[CallableQueue-6] org.apache.oozie.command.wf.ActionEndXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] ERROR is considered as FAILED for SLA
2023-12-22 03:43:40,262 INFO THREAD[CallableQueue-6] org.apache.oozie.service.JPAService: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] No results found
2023-12-22 03:43:40,280 INFO THREAD[CallableQueue-6] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@fail] Start action [0000040-231010100945447-oozie-oozi-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2023-12-22 03:43:40,284 INFO THREAD[CallableQueue-6] org.apache.oozie.action.control.KillActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@fail] Starting action
2023-12-22 03:43:40,288 INFO THREAD[CallableQueue-6] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@fail] [***0000040-231010100945447-oozie-oozi-W@fail***]Action status=DONE
2023-12-22 03:43:40,288 INFO THREAD[CallableQueue-6] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@fail] [***0000040-231010100945447-oozie-oozi-W@fail***]Action updated in DB!
2023-12-22 03:43:40,300 INFO THREAD[CallableQueue-6] org.apache.oozie.action.control.KillActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@fail] Action ended with external status [OK]
2023-12-22 03:43:40,353 INFO THREAD[CallableQueue-5] org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[ln000xhstm0002.federated.fds] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@fail] No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W@fail
2023-12-22 03:43:40,353 INFO THREAD[CallableQueue-8] org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[ln000xhstm0002.federated.fds] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W@spark-node
2023-12-22 03:43:40,353 INFO THREAD[CallableQueue-5] org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[ln000xhstm0002.federated.fds] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W
Created 12-27-2023 06:02 AM
@Raghunath_CDH As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post. Thanks.
Regards,
Diana Torres,