Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

View resources allocated to particular application in Cloudera Manager

View resources allocated to particular application in Cloudera Manager

New Contributor

 

 

Is there a way to view the resources allocated to a particular yarn application while it is running?

 

The issue I am trying to determine is why my spark application decides to shut down the spark context.

 

I've greped the yarn logs for exceptions and I didn't find any memory related issues.

3 REPLIES 3

Re: View resources allocated to particular application in Cloudera Manager

Champion

The RM UI and the Spark History are what you are looking for.  The yarn command can also be used in a pinch.

Re: View resources allocated to particular application in Cloudera Manager

New Contributor
Can you point me to particular documentation?

Ideally it would the same output at Ganglia where you can view I/O, memory, disk for each machine in use.

Re: View resources allocated to particular application in Cloudera Manager

Champion
https://spark.apache.org/docs/1.6.2/monitoring.html

This is old but still useful.

http://hadooptutorial.info/yarn-web-ui/

I know you can get the info from the RM UI. You should be able to get it from the Spark History UI through curl commands as well. The information could then be fed into Ganglia.