JVM Reuse

msumbul — Thu, 22 May 2014 15:07:57 GMT

I would like to know if there is any drawback to set the following setting bigger than one?

•mapred.job.reuse.jvm.num.tasks

Why it set by default to one?

Thanks in advance,

Re: JVM Reuse

Wilfred — Thu, 29 May 2014 03:58:24 GMT

We do not recommend setting it to anything but the default value (1, which means no reuse)

JVM reuse should help with performance becuase it removes the startup lag of the JVM but it is only marginal and comes with a number of drawbacks. Most task will run for a long time (tens of seconds or even minutes) and startup times are not the problem when you look at those run times. When you re-use a JVM there is a change that the heap is not completely clean and it will be fragmented. The fragmentation can lead to more GC's and negate the startup gain. If there is a memory leak it could also affect the memory usage etc.

Wilfred

Re: JVM Reuse

msumbul — Wed, 04 Jun 2014 20:43:30 GMT

Thanks for the answer 🙂

question JVM Reuse in Archives of Support Questions (Read Only)

JVM Reuse

Re: JVM Reuse

Re: JVM Reuse