question Node labels examples in Archives of Support Questions (Read Only)

Node labels examples

sciencex — Sun, 01 May 2016 03:56:53 GMT

Hello,

I configured a yarn cluster to be used with node labels. Could you please provide a simple example by using the "pi" or wordcount applications with node labels. I tried to use "hadoop jar /HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapred*examples.jar pi 10 10 -node_labels_expression node1" but I got an error because of the command is not well formed . Without -node_labels_expression the execution is hanged without errors.

Thanks

Re: Node labels examples

pminovic — Sun, 01 May 2016 07:46:12 GMT

Only Yarn distributedshell supports specifying node labels on the command line. For MR jobs like Pi and wordcount, create a queue, set its default label, and submit your MR job to that queue.

Re: Node labels examples

nmaillard1 — Sun, 01 May 2016 15:17:24 GMT

In order to use node labels you will first have to enable them in yarn: yarn.node-labels.enabled true

then set up a label directory, create labels and associate to hosts and queue. Labels are logically accessed through the capacity queue they are associated with. In your case it would just be running your job in the right yarn capacity queue. The documentation has an example that can help you: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_yarn_resource_mgt/content/configuring_node_labels.html

Re: Node labels examples

nsabharwal — Sun, 01 May 2016 18:22:46 GMT

@arwing mceagle

See this demo https://community.hortonworks.com/articles/11434/yarn-node-labels-1.html