Reply
Highlighted
Explorer
Posts: 23
Registered: ‎02-07-2015

Problem with configuration management?

On Yarn configuration I have set:

Map Task Maximum Heap Size = 2 GiB

Reduce Task Maximum Heap Size = 2GiB

Client Java Heap Size in Bytes = 2Gib

 

Tasks fails with the following dump:

2015-05-22 20:53:32,255 FATAL [AsyncDispatcher event handler] org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread
java.lang.NumberFormatException: For input string: "2147483648"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Integer.parseInt(Integer.java:499)
	at java.lang.Integer.parseInt(Integer.java:527)
	at org.apache.hadoop.mapred.JobConf.parseMaximumHeapSizeMB(JobConf.java:2105)

 

Unfortunately the ...48 is one too much for 32 bits, but shouldn't Cloudera keep units on the string instead of forcing to "bytes" unit just to fail ?

 

 

 

Cloudera Employee
Posts: 508
Registered: ‎07-30-2013

Re: Problem with configuration management?

This is a known issue (MAPREDUCE-6343) which has since been fixed in CDH5.4.1
http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_rn_fixed_in_541.htm...

Thanks,
Darren
Announcements