Node labels examples

[#1]

Hello,

I configured a yarn cluster to be used with node labels. Could you please provide a simple example by using the "pi" or wordcount applications with node labels. I tried to use "hadoop jar /HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapred*examples.jar pi 10 10 -node_labels_expression node1" but I got an error because of the command is not well formed . Without -node_labels_expression the execution is hanged without errors.

Thanks

[#2 · Accepted solution]

Only Yarn distributedshell supports specifying node labels on the command line. For MR jobs like Pi and wordcount, create a queue, set its default label, and submit your MR job to that queue.

[#3]

In order to use node labels you will first have to enable them in yarn: yarn.node-labels.enabled true

then set up a label directory, create labels and associate to hosts and queue. Labels are logically accessed through the capacity queue they are associated with. In your case it would just be running your job in the right yarn capacity queue. The documentation has an example that can help you: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_yarn_resource_mgt/content/configuring_nod...

[#4]

@arwing mceagle

See this demo https://community.hortonworks.com/articles/11434/yarn-node-labels-1.html