Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to identify resources usage by a Tez container / Spark executor

Highlighted

How to identify resources usage by a Tez container / Spark executor

New Contributor

Hello Everyone,

Is there a way we can identify resources wasted or used by Tez container, for example if we set "hive.tez.continer.size=8192" how do we identify how much of memory being used and wasted.

In our environment we have Dr.Elephant but shows resources usage/wastage only for MR jobs as we don't have ATS enabled, is there a way or work-around to monitor resource usage and wastage from Yarn logs or any other way.