Member since
12-27-2023
5
Posts
2
Kudos Received
0
Solutions
02-27-2024
08:06 AM
2 Kudos
Dear Support what kind of case we should open I don't have any information on this issue please respond.
... View more
01-16-2024
04:54 AM
Dear Support any replies are appreciated. Thanks.
... View more
01-08-2024
12:59 AM
Error un-registering AM client java.lang.NullPointerException at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.unregisterApplicationMaster(AMRMClientImpl.java:613) at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.unregisterApplicationMaster(AMRMClientAsyncImpl.java:220) at org.apache.oozie.action.hadoop.LauncherAM.unregisterWithRM(LauncherAM.java:359) at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:285) at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144) Callback notification attempts left 0 Callback notification trying http://ln000xhstm0002.federated.fds:11000/oozie/callback?id=0000046-231010100945447-oozie-oozi-W@spark-node&status=SUCCEEDED Callback notification to http://ln000xhstm0002.federated.fds:11000/oozie/callback?id=0000046-231010100945447-oozie-oozi-W@spark-node&status=SUCCEEDED succeeded Callback notification succeeded
... View more
Labels:
- Labels:
-
Apache Oozie
12-28-2023
09:36 PM
Hi Even I got the same INFO messages and when I tried to get the logs for that Job application ID below are the logs oozie job -log 0000041-231010100945447-oozie-oozi-W -logfilter loglevel=WARN\;limit=10 -oozie http://localhost:11000/oozie/ Last login: Thu Dec 28 03:31:59 2023 from l05017024 [bh24224@localhost ~]$ oozie job -log 0000041-231010100945447-oozie-oozi-W -logfilter loglevel=WARN\;limit=10 -oozie http://localhost:11000/oozie/ 2023-12-27 02:39:01,589 WARN THREAD[CallableQueue-25] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[localhost] USER[fsgapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${ADMIN}-on-${MY_APP}] JOB[0000041-231010100945447-oozie-oozi-W] ACTION[0000041-231010100945447-oozie-oozi-W@spark-node] No credential properties found for action : 0000041-231010100945447-oozie-oozi-W@spark-node, cred : null 2023-12-27 02:39:01,594 WARN THREAD[CallableQueue-25] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[localhost] USER[fsgapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${ADMIN}-on-${MY_APP}] JOB[0000041-231010100945447-oozie-oozi-W] ACTION[0000041-231010100945447-oozie-oozi-W@spark-node] Invalid configuration value [null] defined for launcher max attempts count, using default [2]. 2023-12-27 02:49:08,017 WARN THREAD[CallableQueue-26] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[localhost] USER[fsgapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://localhost/UI] JOB[0000041-231010100945447-oozie-oozi-W] ACTION[0000041-231010100945447-oozie-oozi-W@spark-node] Launcher AM died, check Hadoop LOG for job [localhost:8032:application_1696946942137_0020] Please suggest me is this environment issue or kind of any changes to be done for the job. CC: Cloudera_Support.
... View more
12-27-2023
02:32 AM
Hi All I am new to the CDH and the similar error I am also getting when run Oozie job through Java client application. The error Logs displays like below Main error is " No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W@spark-node" USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[] No results found 2023-12-22 03:33:19,121 INFO THREAD[qtp236002428-766348] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] Start action [0000040-231010100945447-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2023-12-22 03:33:19,125 INFO THREAD[qtp236002428-766348] org.apache.oozie.action.control.StartActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] Starting action 2023-12-22 03:33:19,130 INFO THREAD[qtp236002428-766348] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] [***0000040-231010100945447-oozie-oozi-W@:start:***]Action status=DONE 2023-12-22 03:33:19,131 INFO THREAD[qtp236002428-766348] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] [***0000040-231010100945447-oozie-oozi-W@:start:***]Action updated in DB! 2023-12-22 03:33:19,146 INFO THREAD[qtp236002428-766348] org.apache.oozie.action.control.StartActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] Action ended with external status [OK] 2023-12-22 03:33:19,173 INFO THREAD[qtp236002428-766348] org.apache.oozie.service.JPAService: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] No results found 2023-12-22 03:33:19,186 INFO THREAD[CallableQueue-7] org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[ln000xhstm0002.federated.fds] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W@:start: 2023-12-22 03:33:19,186 INFO THREAD[CallableQueue-37] org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[ln000xhstm0002.federated.fds] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W 2023-12-22 03:33:19,234 INFO THREAD[CallableQueue-7] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Start action [0000040-231010100945447-oozie-oozi-W@spark-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2023-12-22 03:33:19,238 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Starting action. Getting Action File System 2023-12-22 03:33:19,249 INFO THREAD[CallableQueue-7] org.apache.oozie.service.SparkConfigurationService: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] The atlas-application.properties does not exists for *: /etc/spark/conf/atlas-application.properties 2023-12-22 03:33:19,256 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Initializing sharelib excluder for action if specified. 2023-12-22 03:33:19,259 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.ShareLibExcluder: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Sharelib exclude pattern not configured, skipping. 2023-12-22 03:33:19,266 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Initializing sharelib excluder for action if specified. 2023-12-22 03:33:19,266 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.ShareLibExcluder: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Sharelib exclude pattern not configured, skipping. 2023-12-22 03:33:19,282 WARN THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] No credential properties found for action : 0000040-231010100945447-oozie-oozi-W@spark-node, cred : null 2023-12-22 03:33:19,287 WARN THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Invalid configuration value [null] defined for launcher max attempts count, using default [2]. 2023-12-22 03:33:19,287 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.YarnACLHandler: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Not setting ACLs because mapreduce.cluster.acls.enabled is set to false 2023-12-22 03:33:19,652 INFO THREAD[CallableQueue-7] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] checking action, hadoop job ID [application_1696946942137_0018] status [RUNNING] 2023-12-22 03:33:19,654 INFO THREAD[CallableQueue-7] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] [***0000040-231010100945447-oozie-oozi-W@spark-node***]Action status=RUNNING 2023-12-22 03:33:19,654 INFO THREAD[CallableQueue-7] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP[${INDEXER_PARAMS}-ReindexJob-${USER_ROLE}-on-${USER_ENTITY_HOST}] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] [***0000040-231010100945447-oozie-oozi-W@spark-node***]Action updated in DB! 2023-12-22 03:33:19,666 INFO THREAD[CallableQueue-4] org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[ln000xhstm0002.federated.fds] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W@spark-node 2023-12-22 03:43:40,218 INFO THREAD[CallableQueue-6] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] action completed, external ID [application_1696946942137_0018] 2023-12-22 03:43:40,218 WARN THREAD[CallableQueue-6] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Launcher AM died, check Hadoop LOG for job [ln000xhstm0002.federated.fds:8032:application_1696946942137_0018] 2023-12-22 03:43:40,235 INFO THREAD[CallableQueue-6] org.apache.oozie.action.hadoop.SparkActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] Action ended with external status [FAILED/KILLED] 2023-12-22 03:43:40,241 INFO THREAD[CallableQueue-6] org.apache.oozie.command.wf.ActionEndXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] ERROR is considered as FAILED for SLA 2023-12-22 03:43:40,262 INFO THREAD[CallableQueue-6] org.apache.oozie.service.JPAService: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] No results found 2023-12-22 03:43:40,280 INFO THREAD[CallableQueue-6] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@fail] Start action [0000040-231010100945447-oozie-oozi-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2023-12-22 03:43:40,284 INFO THREAD[CallableQueue-6] org.apache.oozie.action.control.KillActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@fail] Starting action 2023-12-22 03:43:40,288 INFO THREAD[CallableQueue-6] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@fail] [***0000040-231010100945447-oozie-oozi-W@fail***]Action status=DONE 2023-12-22 03:43:40,288 INFO THREAD[CallableQueue-6] org.apache.oozie.command.wf.ActionStartXCommand: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@fail] [***0000040-231010100945447-oozie-oozi-W@fail***]Action updated in DB! 2023-12-22 03:43:40,300 INFO THREAD[CallableQueue-6] org.apache.oozie.action.control.KillActionExecutor: SERVER[ln000xhstm0002.federated.fds] USER[myapp] GROUP[-] TOKEN[] APP["UPC"-ReindexJob-ASYNC-on-http://USER_ENTITY-mcom-mycell2.tbe.zeus.fds.com/UI] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@fail] Action ended with external status [OK] 2023-12-22 03:43:40,353 INFO THREAD[CallableQueue-5] org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[ln000xhstm0002.federated.fds] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@fail] No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W@fail 2023-12-22 03:43:40,353 INFO THREAD[CallableQueue-8] org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[ln000xhstm0002.federated.fds] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[0000040-231010100945447-oozie-oozi-W@spark-node] No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W@spark-node 2023-12-22 03:43:40,353 INFO THREAD[CallableQueue-5] org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[ln000xhstm0002.federated.fds] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000040-231010100945447-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000040-231010100945447-oozie-oozi-W
... View more