Support Questions
Find answers, ask questions, and share your expertise

Cluster configuration

Explorer

Can somebody tell me what would be the real time cluster configuration? I have setup Hortonworks in my home system however it is standalone in real time project what would be the cluster configuration like how many nodes, Cluster memory & RAM, Node memory & RAM, backup of cluster and all?

And while submitting Spark job to YARN how can we decide executors, memory and all those properties?

1 ACCEPTED SOLUTION

Accepted Solutions

Guru

Hi @Hardik Dave

Cluster sizing and planning would require much more detail and in-depth conversation about the use case, the data sizing, etc. A good guide that can help you down the path of sizing your cluster can be found here: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_cluster-planning/bk_cluster-planning.pdf

I would suggest using Ambari to manage your cluster where the memory allocation and settings/config will be much more visible in the UI for each technology/service being used in your cluster.

View solution in original post

1 REPLY 1

Guru

Hi @Hardik Dave

Cluster sizing and planning would require much more detail and in-depth conversation about the use case, the data sizing, etc. A good guide that can help you down the path of sizing your cluster can be found here: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_cluster-planning/bk_cluster-planning.pdf

I would suggest using Ambari to manage your cluster where the memory allocation and settings/config will be much more visible in the UI for each technology/service being used in your cluster.

View solution in original post