I have a job which successfully completed 48hrs ago, now i have a request to get total containers, total memory used for this job. How do i get it?
Thanks in advance.
Whatever we collect is in the Job History Server. The counters are there too.
Since each task attempt will be a container you also have the number of containers used.
Thanks for the reply Wilfred. I have following info on Job History server, but not sure how to get 'how many containers allocated', even checked logs.
For each task attempt i can see following
Total Vmem allocated for containers
Total Pmem allocated for containers
Total Vcores allocated for containers
Each task attempt will be a new container. On the front page you see all attempts: failed, killed and succeeded.
Add them all together with the AM attempts number and you get the number of containers that ran.