Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to find which user has killed the yarn application from cli

avatar

how can i check which user has killed application from yarn cli ?

1 REPLY 1

avatar

Hi @Anurag Mishra

Prior to HDP 3 you could only see that an application was killed by a user, not who killed the application
HDP 3 and onwards is more informative about who killed an application.

[jsneep@node4 ~]$ yarn jar /usr/hdp/2.6.4.0-91/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 10 9000

18/07/24 07:44:44 INFO security.TokenCache: Got dt for hdfs://hwc1251-node2.hogwarts-labs.com:8020; Kind: HDFS_DELEGATION_TOKEN, Service: 172.25.33.145:8020, Ident: (HDFS_DELEGATION_TOKEN token 7 for jsneep)
18/07/24 07:44:45 INFO input.FileInputFormat: Total input paths to process : 10
18/07/24 07:44:45 INFO mapreduce.JobSubmitter: number of splits:10
18/07/24 07:44:45 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1532417644227_0005

18/07/24 07:44:46 INFO impl.YarnClientImpl: Submitted application application_1532417644227_0005
18/07/24 07:44:46 INFO mapreduce.Job: Running job: job_1532417644227_0005
[root@hwc1251-node4 ~]# yarn application -kill application_1532417644227_0005
18/07/24 07:44:53 INFO mapreduce.Job: Job job_1532417644227_0005 failed with state KILLED due to: Application killed by user.
18/07/24 07:44:53 INFO mapreduce.Job: Counters: 0
Job Finished in 8.516 seconds

Ex, above I've submitted a yarn job (application_1532417644227_0005) & killed it. The logs state "Application killed by user."

I can also browse the Resource Manager UI at http://<RM IP ADDRESS>:8088/cluster/apps/KILLED and see that it was killed by a user.


The apache jira for this:
https://issues.apache.org/jira/browse/YARN-5053 | "More informative diagnostics when applications killed by a user"

In my HDP3 cluster, when I submit an identical job and kill it;

[root@c2175-node4 ~]# yarn app -kill application_1532419910561_0001
18/07/24 08:12:45 INFO client.RMProxy: Connecting to ResourceManager at c2175-node2.hwx.com/172.25.39.144:8050
18/07/24 08:12:45 INFO client.AHSProxy: Connecting to Application History server at c2175-node2.hwx.com/172.25.39.144:10200
Killing application application_1532419910561_0001

For example now via the RM UI, I can browse to http://<RM IP ADDRESS>:8088/ui2/#/yarn-app/application_x_/info and under diagnostics we will see the user and source address of the kill operation. The same would be visible through CLI, via "yarn app -status application_1532419910561_0001 | grep killed"

Application application_1532419910561_0001 was killed by user root at 172.25.33.15

Edit: PS, you could make use of YARN Queues & ACLs to limit / determine who has rights to kill yarn applications, I wanted to mention this in case you're looking for something to help you if you're currently unable to get your cluster upgraded to HDP3.
Further info: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_yarn-resource-management/content/control...


*please mark this answer as accepted if you found this helpful*