question Re: Can't set mapreduce.shuffle.max.connections to 0 in Archives of Support Questions (Read Only)

Can't set mapreduce.shuffle.max.connections to 0

jiro — Fri, 16 Sep 2022 09:10:29 GMT

Hi,

I tried to set mapreduce.shuffle.max.connections to 0, but a validation error occured.

The doc says:

Set to 0 (zero) to indicate no limit on the number of connections.

Error: 0 is less than the minimum allowed value 1.

Please see the attached screenshot.

Version: Cloudera Enterprise Data Hub Edition Trial 5.2.0 (#60 built by jenkins on 20141012-2239 git: 179000584849e68f98ad2a7fe710723bd6c29c98)

CDH: 5.1.2

Regards,

Re: Can't set mapreduce.shuffle.max.connections to 0

Anthony — Fri, 29 May 2015 17:43:22 GMT

Hi Jiro,

Thanks for bringing this to our attention. It appears there's a bug with the validation for that property, in which the lower bound should be 0 and not 1. We'll get this fixed in a future release. In the meantime, you can accomplish setting mapreduce.shuffle.max.connections to 0 by setting it in a specific Safety Valve in Cloudera Manager. Please follow the directions listed below:

Login to CM with admin privileges if you haven't done so already.
Navigate to the YARN configuration page (CM -> Clusters -> YARN -> Configuration)
Naviate to Gateway Default Group -> Advanced -> MapReduce Client Advanced Configuration Snippet (Safety Valve) for mapred-site.xml
In the field marked Value, please enter in the following entry:

<property>
<name>mapreduce.shuffle.max.connections</name>
<value>0</value>
</property>

5. Save changes then Deploy client configuration

Kind Regards,

Anthony

Re: Can't set mapreduce.shuffle.max.connections to 0

jiro — Sat, 30 May 2015 05:21:03 GMT

Thanks!