Reply
Explorer
Posts: 7
Registered: ‎02-20-2014
Accepted Solution

JVM Reuse

Hi

 

I would like to know if there is any drawback to set the following setting bigger than one?

•mapred.job.reuse.jvm.num.tasks
 
Why it set by default to one?
 
Thanks in advance,
Cloudera Employee
Posts: 290
Registered: ‎01-16-2014

Re: JVM Reuse

We do not recommend setting it to anything but the default value (1, which means no reuse)

JVM reuse should help with performance becuase it removes the startup lag of the JVM but it is only marginal and comes with a number of drawbacks. Most task will run for a long time (tens of seconds or even minutes) and startup times are not the problem when you look at those run times. When you re-use a JVM there is a change that the heap is not completely clean and it will be fragmented. The fragmentation can lead to more GC's and negate the startup gain. If there is a memory leak it could also affect the memory usage etc.

 

Wilfred

Highlighted
Explorer
Posts: 7
Registered: ‎02-20-2014

Re: JVM Reuse

Thanks for the answer :)

Announcements