Reply
Posts: 1,491
Kudos: 246
Solutions: 226
Registered: ‎07-31-2013

Re: Where is the setting for the port-range used by org.apache.hadoop.mapred.YarnChild

The property is not server side, its per-app (client side). All MR2
properties (save for shuffle) are client properties in YARN.
Backline Customer Operations Engineer
Expert Contributor
Posts: 68
Registered: ‎10-04-2016

Re: Where is the setting for the port-range used by org.apache.hadoop.mapred.YarnChild?

Could you check if this JIRA is fixed by Cloudera?

https://issues.apache.org/jira/browse/MAPREDUCE-6338

Expert Contributor
Posts: 68
Registered: ‎10-04-2016

Re: Where is the setting for the port-range used by org.apache.hadoop.mapred.YarnChild

Thanks for your quick response. I did the quick test after putting the client side setting, but it still doesn't work. I still saw MR job failed due to No Route to Host from slave 1 to slave 2 on port not defined within the set range.

Posts: 1,491
Kudos: 246
Solutions: 226
Registered: ‎07-31-2013

Re: Where is the setting for the port-range used by org.apache.hadoop.mapred.YarnChild

[ Edited ]

Have you already ensured your launched app's specific configuration page on the JobHistory reflects the changed configs? Did you ensure to also deploy the client config change via a cluster-wide redeploy? https://www.youtube.com/watch?v=4S9H3wftM_0

I'm able to limit the ports just fine. The test case even passes.

 

Given you appear to have pre-limited iptables rules causing a NoRouteToHost for the IPCs between client and MR2 AM, have you already ensured that among the open port range you are able to make a proper connection by running something outside of CDH such as a simple Python HTTP service (python -m SimpleHttpServer -p some_target_open_range_port) on all NodeManagers and connecting to them from the edge host?

Backline Customer Operations Engineer
Expert Contributor
Posts: 68
Registered: ‎10-04-2016

Re: Where is the setting for the port-range used by org.apache.hadoop.mapred.YarnChild

[ Edited ]

Redeployed the client configuration from CM.  Checked both yarn and hive configuration, both mapred-site.xml files have the correct configuration reflected.

Expert Contributor
Posts: 68
Registered: ‎10-04-2016

Re: Where is the setting for the port-range used by org.apache.hadoop.mapred.YarnChild

[ Edited ]

I am not using iptables, using firewalld instead on Centos 7.x.   The error I saw is caused by Hive doing select count(*) on a table, and the log indicates that the communication is between two slave nodes, not between edge node and AM.

Please check your test setting. If your range is large enough, some jobs might succeed. With my current setting of 6000 port range, some jobs failed, and some succeeded by retrying and hitting the port within the range.

 

New Contributor
Posts: 3
Registered: ‎05-06-2015

Re: Where is the setting for the port-range used by org.apache.hadoop.mapred.YarnChild

I am in the same boat - we have a restrictive firewall in place and I am trying to open a range of ports 49900 - 50000 with the following in the mapred-site.xml.

<property>

    <name>yarn.app.mapreduce.am.job.client.port-range</name>

    <value>49900-50000</value>

  </property>

 

I am not able to restrict the ports at all - I see the following when I run my job - 

Got exception: java.net.NoRouteToHostException: No Route to Host from  rm.domain.com/X.X.X.XXXX to workernodeX.domain.com:38470 - AFAIK - No route to host means the destination firewall is kicking me out.

 

I am on cloudera CDH 5.10.0 - does it include the below fix forhttps://issues.apache.org/jira/browse/MAPREDUCE-6338? If not which version would - I need to run this thing with extensive firewall in place and hence the question.

 

 

Thanks in advance for your help!

Announcements