Support Questions

Find answers, ask questions, and share your expertise

How to find which user has killed the yarn application from cli

avatar

how can i check which user has killed application from yarn cli ?

1 REPLY 1

avatar

Hi @Anurag Mishra

Prior to HDP 3 you could only see that an application was killed by a user, not who killed the application
HDP 3 and onwards is more informative about who killed an application.

[jsneep@node4 ~]$ yarn jar /usr/hdp/2.6.4.0-91/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 10 9000

18/07/24 07:44:44 INFO security.TokenCache: Got dt for hdfs://hwc1251-node2.hogwarts-labs.com:8020; Kind: HDFS_DELEGATION_TOKEN, Service: 172.25.33.145:8020, Ident: (HDFS_DELEGATION_TOKEN token 7 for jsneep)
18/07/24 07:44:45 INFO input.FileInputFormat: Total input paths to process : 10
18/07/24 07:44:45 INFO mapreduce.JobSubmitter: number of splits:10
18/07/24 07:44:45 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1532417644227_0005

18/07/24 07:44:46 INFO impl.YarnClientImpl: Submitted application application_1532417644227_0005
18/07/24 07:44:46 INFO mapreduce.Job: Running job: job_1532417644227_0005
[root@hwc1251-node4 ~]# yarn application -kill application_1532417644227_0005
18/07/24 07:44:53 INFO mapreduce.Job: Job job_1532417644227_0005 failed with state KILLED due to: Application killed by user.
18/07/24 07:44:53 INFO mapreduce.Job: Counters: 0
Job Finished in 8.516 seconds

Ex, above I've submitted a yarn job (application_1532417644227_0005) & killed it. The logs state "Application killed by user."

I can also browse the Resource Manager UI at http://<RM IP ADDRESS>:8088/cluster/apps/KILLED and see that it was killed by a user.


The apache jira for this:
https://issues.apache.org/jira/browse/YARN-5053 | "More informative diagnostics when applications killed by a user"

In my HDP3 cluster, when I submit an identical job and kill it;

[root@c2175-node4 ~]# yarn app -kill application_1532419910561_0001
18/07/24 08:12:45 INFO client.RMProxy: Connecting to ResourceManager at c2175-node2.hwx.com/172.25.39.144:8050
18/07/24 08:12:45 INFO client.AHSProxy: Connecting to Application History server at c2175-node2.hwx.com/172.25.39.144:10200
Killing application application_1532419910561_0001

For example now via the RM UI, I can browse to http://<RM IP ADDRESS>:8088/ui2/#/yarn-app/application_x_/info and under diagnostics we will see the user and source address of the kill operation. The same would be visible through CLI, via "yarn app -status application_1532419910561_0001 | grep killed"

Application application_1532419910561_0001 was killed by user root at 172.25.33.15

Edit: PS, you could make use of YARN Queues & ACLs to limit / determine who has rights to kill yarn applications, I wanted to mention this in case you're looking for something to help you if you're currently unable to get your cluster upgraded to HDP3.
Further info: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_yarn-resource-management/content/control...


*please mark this answer as accepted if you found this helpful*