Node label support in CDH 5.4.x?

[#1]

Hi,

Two questions about node label support in CDH 5.4.x.

1. The docs for CDH 5.4.x make no mention of node labels that I can find.

Since 5.4.x includes Apache Hadoop 2.6 which supports node labels I would expect they are supported, but can find no official statement.

Are node labels supported in CDH 5.4.x?

2. Are node labels supported in the Fair Scheduler or just the Capacity Scheduler?

Based on this JIRA it would seem I can't make use of node labels if I use the Fair Scheduler.

https://issues.apache.org/jira/browse/YARN-2497

Is there any way around this?

Thanks,

Eric

[#2 · Accepted solution]

(1) - While the CDH codebase does carry the 2.6 initial node-label implementation features, a lot many more changes and enhancements for node-labelling made it upstream only to 2.8, so its a feature still under some development. You can utilise the 2.6 features for certain in CDH 5.4.x, but only via the CapacityScheduler (following the upstream docs), cause the code support does exist in the sources: https://github.com/cloudera/hadoop-common/tree/cdh5.4.7-release/

(2) FairScheduler support is not in upstream yet. We do have node-labelling for FairScheduler on our roadmap for a future release, but I don't have a shareable ETA for it yet.

[#3]

Any update on when node labels will be available?  I don't see it in CDH 5.5.1

[#4]

I am looking for node label support in CDH 5.7.1. Please let me know if there is any progress and documentation for it's implementation.

[#5]

Hi,

Any update on node label support for FairScheduler in some future CDH version?

Thanks.

Regards,

Iván