Reply
Contributor
Posts: 28
Registered: ‎05-12-2015

Fair scheduler pools mismatch

Hello guys, we are running CDH 4.7.1 with CM 4.8.5 and we have pools set in fair schedular using CM's mapreduce configurations. 

 

One of the pools is set with following thresholds 

 

<pool name="pool-name">
    <minMaps>144</minMaps>
    <minReduces>30</minReduces>
    <maxMaps>144</maxMaps>
    <maxReduces>30</maxReduces>​
  </pool>
 
 
But the schedular page at 'jt-host:50030/scheduler' show a completely different value:
 
Screen Shot 2015-06-25 at 11.26.59 AM.png
 
There are many other pools set but only this particular pool has the issue.
 
Could you guys suggest any ways to debug this issue ?
 
- Thanks
vganji
Engineer
Highlighted
Cloudera Employee
Posts: 285
Registered: ‎01-16-2014

Re: Fair scheduler pools mismatch

Can you compare that information to the JT web UI and check the cluster summary and the number of occupied slots etc.

There is also a FS eventlog which can show you what the scheduler is doing. See the really old hadoop fair scheduler information.

You do know that CDH 4 is almost EOL and that you should move to CDH 5 and YARN and are planning for that (I hope).

 

Wilfred

Announcements