Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

asymmetric cluster setup (compute and storage done in separate datanodes by labeling them)

asymmetric cluster setup (compute and storage done in separate datanodes by labeling them)

Hi,

I would like to setup an asymmetric cluster with 6 datanodes. I labelled them as

datanode1, datanode2, datanode3-compute

datanode4, datanode5, datanode6-storage

using this command

yarn rmadmin -replaceLabelsOnNode "data1.aws.asymmetric.com:45454=compute data2.aws.asymmetric.com:45454=compute data3.aws.asymmetric.com:45454=compute data4.aws.asymmetric.com:45454=storage data5.aws.asymmetric.com:45454=storage data6.aws.asymmetric.com:45454=storage"

then changed the yarn configuration, so that the queue(default) can be accessed only by the compute nodes

yarn.scheduler.capacity.root.queues=default

yarn.scheduler.capacity.root.default.capacity=100

yarn.scheduler.capacity.root.default.accessible-node-labels=compute

yarn.scheduler.capacity.root.default.accessible-node-labels.compute.capacity=100

How to check if the compute nodes only performs computation and that storage nodes stores only the data.

1 REPLY 1
Highlighted

Re: asymmetric cluster setup (compute and storage done in separate datanodes by labeling them)

Don't have an account?
Coming from Hortonworks? Activate your account here