Support Questions
Find answers, ask questions, and share your expertise

Cluster configuration for compute heavy workload and lotsa Spark jobs predicted.

Cluster configuration for compute heavy workload and lotsa Spark jobs predicted.

Explorer

65012-capture.png

Does this server configuration hold well for dev and production environments especially when we are going to have lotsa spark jobs which are resource intensive. Also we are expecting lot of network traffic so should we be boosting the network from 1GB to a bigger size. Any input would be greatly appreciated.

1 REPLY 1

Re: Cluster configuration for compute heavy workload and lotsa Spark jobs predicted.

Guru

@Guru Kamath You should refer to both HDP recommendations and especially http://spark.apache.org/docs/latest/hardware-provisioning.html