Pyspark yarn-client application lifetime

@Jaaaaay Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Spark experts @Bharati @jagadeesan  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Admins can enforce application lifetime SLAs at a service level by setting yarn.scheduler.capacity.<queue-path>.maximum-application-lifetime ” and “yarn.scheduler.capacity.root.<queue-path>.default-application-lifetime” in
capacity-scheduler.xml.
CM > Yarn > Configuration > Capacity Scheduler Configuration Advanced Configuration Snippet (Safety Valve)

reference link is https://blog.cloudera.com/enforcing-application-lifetime-slas-yarn/

How are you setting this parameter? Can you share the entire stacktrace of the failure?

@Bharati Any updates? I have tried the REST API method. Getting my app’s expiry time works just fine. But put request get a 401 unauthorized error.  It’s a shared cluster and I don’t have the admin-level authorization.