Reply
Explorer
Posts: 6
Registered: ‎07-27-2015

How to get containers used info for the job successfully finished 2 days ago?, I have job ID details

I have a job which successfully completed 48hrs ago, now i have a request to get total containers, total memory used for this job. How do i get it?

 

Thanks in advance. 

 

Asif

Cloudera Employee
Posts: 306
Registered: ‎01-16-2014

Re: How to get containers used info for the job successfully finished 2 days ago?, I have job ID det

Whatever we collect is in the Job History Server. The counters are there too.

Since each task attempt will be a container you also have the number of containers used.

 

Wilfred

Explorer
Posts: 6
Registered: ‎07-27-2015

Re: How to get containers used info for the job successfully finished 2 days ago?, I have job ID det

Thanks for the reply Wilfred. I have following info on Job History server, but not sure how to get 'how many containers allocated', even checked logs. 

 

For each task attempt i can see following

 

Total Vmem allocated for containers

Total Pmem allocated for containers

Total Vcores allocated for containers

 

Thanks

Asif

 

Cloudera Employee
Posts: 306
Registered: ‎01-16-2014

Re: How to get containers used info for the job successfully finished 2 days ago?, I have job ID det

Each task attempt will be a new container. On the front page you see all attempts: failed, killed and succeeded.

Add them all together with the AM attempts number and you get the number of containers that ran.

 

Wilfred

Announcements