question Re: How to gracefully stop YARN role on a datanode CDH 6.3.2 in Support Questions

How to gracefully stop YARN role on a datanode CDH 6.3.2

xpouser — Thu, 23 Dec 2021 15:22:56 GMT

Hi All,

I'm new person is a hadoop world.

Can not find answer on this question.

How to gracefully stop YARN role on a data node and save or pause running jobs

So the jobs will not exit with killer or crash status.

I know that in ClouderaManager you can decommission yarn role when you can stop it.

Is this a safe way to keep the jobs running and not failed state?

Is this a gracefull yarn role shutdown or where is other way to do this?

Please advice me how to do this.

Thank you.

Re: How to gracefully stop YARN role on a datanode CDH 6.3.2

xpouser — Mon, 27 Dec 2021 12:53:10 GMT

I made YARM role decommission on a data node.

And now I see map reduce jobs on that data node have status KILLED.

So My guess is "yarn role decommission" is not a graceful shutdown solution.

Please advice how to do propriety YARN role graceful shutdown on a data node.

Re: How to gracefully stop YARN role on a datanode CDH 6.3.2

GangWar — Fri, 31 Dec 2021 17:22:20 GMT

@xpouser You want to take a look at : https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/GracefulDecommission.html

Re: How to gracefully stop YARN role on a datanode CDH 6.3.2

Amithsha — Sun, 16 Jan 2022 07:32:31 GMT

HI,
YARN Graceful decommission will wait for jobs to complete. You can pass the timeout value so that YARN will start decommission after x seconds. If no jobs running within x secs then automatically YARN will start decommission without waiting for timeout to happen.

CM -> Clusters -> yarn -> Configuration -> In search bar (

yarn.resourcemanager.nodemanager-graceful-decommission-timeout-secs)

Set the value and save the configuration and do restart to deploy configs.

To decommission a specific host/more hosts

CM -> Clusters -> yarn -> Instances (Select the hosts that you want to decommission)

Click -> Actions for selected hosts -> Decommission
In case you want to decommission all the roles of a host then follow this doc
https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_mc_host_maint.html#decomm_host

Make sure to mark the answer as the accepted solution. If it resolves you issue !