Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Node Label Configuration with HDP 2.3.x and Ambari 2.1.x

Solved Go to solution
Highlighted

Node Label Configuration with HDP 2.3.x and Ambari 2.1.x

Contributor

Does anyone have (or can you point me to) a link that explains the steps of using YARN node labels in HDP 2.3 being managed by Ambari 2.1? Our existing documentation is for manual configuration only. I've clicked the enabled button, and have already figured out that Ambari creates slightly different directory structures, etc. than are listed. I can keep going with trial and error, but if someone has already documented the process, caveats, etc. that would be fantastic. Thanks!

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Node Label Configuration with HDP 2.3.x and Ambari 2.1.x

@jeden@hortonworks.com

Thanks for asking this question. I could not find any specific docs and looks like it's straight forward from Ambari end ( as you said it already)

Under Yarn --> Config ( 2 features, Node label and Pre-emption - not relevant to this question)

Link

enable label - yarn.node-labels.enabled

349-screen-shot-2015-10-29-at-42348-pm.png

371-screen-shot-2015-10-29-at-43215-pm.png

Doc mentioned to configure root-dir to HDFS but ambari has above location by default. We can follow the manual guide once enabling the labels through Ambari (Based on my testing in lab)

View solution in original post

6 REPLIES 6
Highlighted

Re: Node Label Configuration with HDP 2.3.x and Ambari 2.1.x

@jeden@hortonworks.com

Thanks for asking this question. I could not find any specific docs and looks like it's straight forward from Ambari end ( as you said it already)

Under Yarn --> Config ( 2 features, Node label and Pre-emption - not relevant to this question)

Link

enable label - yarn.node-labels.enabled

349-screen-shot-2015-10-29-at-42348-pm.png

371-screen-shot-2015-10-29-at-43215-pm.png

Doc mentioned to configure root-dir to HDFS but ambari has above location by default. We can follow the manual guide once enabling the labels through Ambari (Based on my testing in lab)

View solution in original post

Highlighted

Re: Node Label Configuration with HDP 2.3.x and Ambari 2.1.x

Contributor

Again, with caveats. For example, the sample command given to test the settings says to run it as yarn, however it failed for me as some permissions now apparently required the hdfs user.

My bottom line, which would be nice to validate, is this:

1) The command line setup of users and folders is unnecessary, and Ambari sets them up slightly differently than our docs

2) Permissions are also different for folders (yarn hadoop vs. yarn yarn...)

3) The sample hadoop jar command to test (which I ran as yarn jar) requires switching to hdfs rather than yarn.

Does that sound about right?

Highlighted

Re: Node Label Configuration with HDP 2.3.x and Ambari 2.1.x

Contributor

Also - is there a quicker / easier / cleaner way to test the node labels that you used other than the one listed in the docs?

Highlighted

Re: Node Label Configuration with HDP 2.3.x and Ambari 2.1.x

Highlighted

Re: Node Label Configuration with HDP 2.3.x and Ambari 2.1.x

Expert Contributor

I configured 4 node labels, for each node label i assigned two nodes. For each node label, i assigned queue. I submitted an application through slider, it is running on specific queue under specific node labels and nodes.

1292-nlabels.jpg

Now, we are in the process of blowing up the cluster thats why it is showing in orange color and unable to obtain node label information. It will show green color and no error message. I checked in Resource Manager Web UI, it is running under specific nodes.

Highlighted

Re: Node Label Configuration with HDP 2.3.x and Ambari 2.1.x

Expert Contributor

I followed hortonwork's documentation for creating node labels and assigning nodes to node labels.

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_yarn_resource_mgt/content/configuring_nod...

Don't have an account?
Coming from Hortonworks? Activate your account here