Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Node label support in CDH 5.4.x?

avatar
New Contributor

Hi,

 

Two questions about node label support in CDH 5.4.x.

 

1. The docs for CDH 5.4.x make no mention of node labels that I can find.

Since 5.4.x includes Apache Hadoop 2.6 which supports node labels I would expect they are supported, but can find no official statement.

Are node labels supported in CDH 5.4.x?

 

2. Are node labels supported in the Fair Scheduler or just the Capacity Scheduler?

Based on this JIRA it would seem I can't make use of node labels if I use the Fair Scheduler.

https://issues.apache.org/jira/browse/YARN-2497

Is there any way around this?

 

Thanks,

Eric

1 ACCEPTED SOLUTION

avatar
Mentor
(1) - While the CDH codebase does carry the 2.6 initial node-label implementation features, a lot many more changes and enhancements for node-labelling made it upstream only to 2.8, so its a feature still under some development. You can utilise the 2.6 features for certain in CDH 5.4.x, but only via the CapacityScheduler (following the upstream docs), cause the code support does exist in the sources: https://github.com/cloudera/hadoop-common/tree/cdh5.4.7-release/

(2) FairScheduler support is not in upstream yet. We do have node-labelling for FairScheduler on our roadmap for a future release, but I don't have a shareable ETA for it yet.

View solution in original post

4 REPLIES 4

avatar
Mentor
(1) - While the CDH codebase does carry the 2.6 initial node-label implementation features, a lot many more changes and enhancements for node-labelling made it upstream only to 2.8, so its a feature still under some development. You can utilise the 2.6 features for certain in CDH 5.4.x, but only via the CapacityScheduler (following the upstream docs), cause the code support does exist in the sources: https://github.com/cloudera/hadoop-common/tree/cdh5.4.7-release/

(2) FairScheduler support is not in upstream yet. We do have node-labelling for FairScheduler on our roadmap for a future release, but I don't have a shareable ETA for it yet.

avatar
New Contributor

Any update on when node labels will be available?  I don't see it in CDH 5.5.1

avatar
New Contributor

I am looking for node label support in CDH 5.7.1. Please let me know if there is any progress and documentation for it's implementation.

 

 

avatar

Hi,

Any update on node label support for FairScheduler in some future CDH version?

Thanks.

 

Regards,

Iván