Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Problem with configuration management?

Highlighted

Problem with configuration management?

Explorer

On Yarn configuration I have set:

Map Task Maximum Heap Size = 2 GiB

Reduce Task Maximum Heap Size = 2GiB

Client Java Heap Size in Bytes = 2Gib

 

Tasks fails with the following dump:

2015-05-22 20:53:32,255 FATAL [AsyncDispatcher event handler] org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread
java.lang.NumberFormatException: For input string: "2147483648"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Integer.parseInt(Integer.java:499)
	at java.lang.Integer.parseInt(Integer.java:527)
	at org.apache.hadoop.mapred.JobConf.parseMaximumHeapSizeMB(JobConf.java:2105)

 

Unfortunately the ...48 is one too much for 32 bits, but shouldn't Cloudera keep units on the string instead of forcing to "bytes" unit just to fail ?

 

 

 

1 REPLY 1

Re: Problem with configuration management?

This is a known issue (MAPREDUCE-6343) which has since been fixed in CDH5.4.1
http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_rn_fixed_in_541.htm...

Thanks,
Darren