Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Please see the Cloudera blog for information on the Cloudera Response to CVE-2021-4428

asymmetric cluster setup (compute and storage done in separate datanodes by labeling them)

Hi,

I would like to setup an asymmetric cluster with 6 datanodes. I labelled them as

datanode1, datanode2, datanode3-compute

datanode4, datanode5, datanode6-storage

using this command

yarn rmadmin -replaceLabelsOnNode "data1.aws.asymmetric.com:45454=compute data2.aws.asymmetric.com:45454=compute data3.aws.asymmetric.com:45454=compute data4.aws.asymmetric.com:45454=storage data5.aws.asymmetric.com:45454=storage data6.aws.asymmetric.com:45454=storage"

then changed the yarn configuration, so that the queue(default) can be accessed only by the compute nodes

yarn.scheduler.capacity.root.queues=default

yarn.scheduler.capacity.root.default.capacity=100

yarn.scheduler.capacity.root.default.accessible-node-labels=compute

yarn.scheduler.capacity.root.default.accessible-node-labels.compute.capacity=100

How to check if the compute nodes only performs computation and that storage nodes stores only the data.

1 REPLY 1