Support Questions

Find answers, ask questions, and share your expertise

Node labels examples

avatar
New Contributor

Hello,

I configured a yarn cluster to be used with node labels. Could you please provide a simple example by using the "pi" or wordcount applications with node labels. I tried to use "hadoop jar /HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapred*examples.jar pi 10 10 -node_labels_expression node1" but I got an error because of the command is not well formed . Without -node_labels_expression the execution is hanged without errors.

Thanks

1 ACCEPTED SOLUTION

avatar
Master Guru

Only Yarn distributedshell supports specifying node labels on the command line. For MR jobs like Pi and wordcount, create a queue, set its default label, and submit your MR job to that queue.

View solution in original post

3 REPLIES 3

avatar
Master Guru

Only Yarn distributedshell supports specifying node labels on the command line. For MR jobs like Pi and wordcount, create a queue, set its default label, and submit your MR job to that queue.

avatar

In order to use node labels you will first have to enable them in yarn: yarn.node-labels.enabled true

then set up a label directory, create labels and associate to hosts and queue. Labels are logically accessed through the capacity queue they are associated with. In your case it would just be running your job in the right yarn capacity queue. The documentation has an example that can help you: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_yarn_resource_mgt/content/configuring_nod...

avatar
Master Mentor