Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to know which Node is Driver Node, which Node is Worker Node for Spark in Cloudera?

avatar
Contributor

I'm using a tool in which I have to point out the master node (driver node) of the Cloudera Spark Cluster (spark :// <some-spark-master> : 7077). Also as I learned, Spark has "Master Node", "Driver Node" and "Worker Nodes".

So I decided to go to the Cloudera Web Manager and checked the Configuration Tab of the Spark service, but all I found are "Gateway instance" and "History Server instance". Where are the "Driver instance" and "Worker instance"? I can't add these two instances in the "Add Role Instances" too

quangbilly79_0-1675302416028.png

 

 

My guess is that it's in Yarn service configuration, but I can't find anything related to "Master", "Driver" or "Worker" either.

quangbilly79_1-1675302416366.png

 

 

So what is the link to "Spark Master" that ends with 7077 (what is the Node)? I can't find it anywhere in the Configuration tab

1 ACCEPTED SOLUTION

avatar
Master Collaborator
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
1 REPLY 1

avatar
Master Collaborator
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login