Created 02-08-2017 07:51 PM
Any suggestions/script to kill yarn application if it is running more than 20 minutes?
Created 02-08-2017 08:51 PM
Soon with the application lifetime feature (https://issues.apache.org/jira/browse/YARN-3813) you wouldn't have to write such scripts. You would simply set lifetime of the app to 20 mins during creation and it would kill itself on the 20th min mark.
Until then, something like this script might help (assuming you have access to yarn command line). This script can be easily enhanced to run as a cronjob if required, where say it will trigger every 1 min and look for specific apps which has crossed certain lifetime and kill them.
Hope this helps -
#!/bin/bash if [ "$#" -lt 2 ]; then echo "Usage: $0 <app_id> <max_life_in_mins>" exit 1 fi finish_time=`yarn application -status $1 2>/dev/null | grep "Finish-Time" | awk '{print $NF}'` if [ $finish_time -ne 0 ]; then echo "App $1 is not running" exit 1 fi time_diff=`date +%s`-`yarn application -status $1 2>/dev/null | grep "Start-Time" | awk '{print $NF}' | sed 's!$!/1000!'` time_diff_in_mins=`echo "("$time_diff")/60" | bc` echo "App $1 is running for $time_diff_in_mins min(s)" if [ $time_diff_in_mins -gt $2 ]; then echo "Killing app $1" echo yarn application -kill $1 else echo "App $1 should continue to run" fi
Created 02-08-2017 08:51 PM
Soon with the application lifetime feature (https://issues.apache.org/jira/browse/YARN-3813) you wouldn't have to write such scripts. You would simply set lifetime of the app to 20 mins during creation and it would kill itself on the 20th min mark.
Until then, something like this script might help (assuming you have access to yarn command line). This script can be easily enhanced to run as a cronjob if required, where say it will trigger every 1 min and look for specific apps which has crossed certain lifetime and kill them.
Hope this helps -
#!/bin/bash if [ "$#" -lt 2 ]; then echo "Usage: $0 <app_id> <max_life_in_mins>" exit 1 fi finish_time=`yarn application -status $1 2>/dev/null | grep "Finish-Time" | awk '{print $NF}'` if [ $finish_time -ne 0 ]; then echo "App $1 is not running" exit 1 fi time_diff=`date +%s`-`yarn application -status $1 2>/dev/null | grep "Start-Time" | awk '{print $NF}' | sed 's!$!/1000!'` time_diff_in_mins=`echo "("$time_diff")/60" | bc` echo "App $1 is running for $time_diff_in_mins min(s)" if [ $time_diff_in_mins -gt $2 ]; then echo "Killing app $1" echo yarn application -kill $1 else echo "App $1 should continue to run" fi