Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to find out how many worker nodes are there

How to find out how many worker nodes are there

Explorer

I have a CDH cluster with below configuration. Spark gateway is installed on all machines. Each node has 22 cores

2 name nodes

3 Kafka nodes

4 data nodes

1 node to run mysql content store

 

I am trying to find out how many worker/slave nodes are assigned to run spark applications in the cluster. When I see the hadoop UI it shows total vcores assinged as 88. Does that mean only 4 data nodes are configured as worker nodes ? Where and how can I see what all worker/slave nodes are configured.

We are using YARN as resource manager.

 

thanks

RM

1 REPLY 1
Highlighted

Re: How to find out how many worker nodes are there

Expert Contributor

Hi @mranjank 

 

In your setup, your 4 data nodes are your worker nodes. HDFS i.e. where the data are stored is on the data nodes and because of data locality, this is where your 'compute' or 'worker' nodes run too. 

 

If you log into Cloudera Manager and click 'Hosts' at the top and then 'Roles' on the dropdown you will see what services are running on what nodes.

 

I hope that helps.

Steve

Don't have an account?
Coming from Hortonworks? Activate your account here