Can't set mapreduce.shuffle.max.connections to 0

[#1]

Hi,

I tried to set mapreduce.shuffle.max.connections to 0, but a validation error occured.

The doc says:

Set to 0 (zero) to indicate no limit on the number of connections.

Error: 0 is less than the minimum allowed value 1.

Please see the attached screenshot.

Version: Cloudera Enterprise Data Hub Edition Trial 5.2.0 (#60 built by jenkins on 20141012-2239 git: 179000584849e68f98ad2a7fe710723bd6c29c98)

CDH: 5.1.2

Regards,

[#2 · Accepted solution]

Hi Jiro,

Thanks for bringing this to our attention.  It appears there's a bug with the validation for that property, in which the lower bound should be 0 and not 1.  We'll get this fixed in a future release.  In the meantime, you can accomplish setting mapreduce.shuffle.max.connections to 0 by setting it in a specific Safety Valve in Cloudera Manager.  Please follow the directions listed below:

1. Login to CM with admin privileges if you haven't done so already.
2. Navigate to the YARN configuration page (CM -> Clusters -> YARN -> Configuration)
3. Naviate to Gateway Default Group -> Advanced -> MapReduce Client Advanced Configuration Snippet (Safety Valve) for mapred-site.xml
4. In the field marked Value, please enter in the following entry:

<property>
<name>mapreduce.shuffle.max.connections</name>
<value>0</value>
</property>

5. Save changes then Deploy client configuration

Kind Regards,

Anthony

[#3]

Thanks!