memory usage by mapreduce job varies

[#1]

Hello All,

I observed in our cluster environment that one job is taking 100gb of memory and sometimes it is taking 2gb of memory(even though memory is available).

Please suggest.

Thanks,

Priya

[#2]

Is it mapreduce or Spark?

Mapreduce can vary since it's depend on number of mappers and reducers and mainly mappers.

[#3]

Hi Fawze,

It's mapreduce, thanks for inputs.