Created 08-01-2017 06:35 AM
Can somebody tell me what would be the real time cluster configuration? I have setup Hortonworks in my home system however it is standalone in real time project what would be the cluster configuration like how many nodes, Cluster memory & RAM, Node memory & RAM, backup of cluster and all?
And while submitting Spark job to YARN how can we decide executors, memory and all those properties?
Created 08-01-2017 02:38 PM
Hi @Hardik Dave
Cluster sizing and planning would require much more detail and in-depth conversation about the use case, the data sizing, etc. A good guide that can help you down the path of sizing your cluster can be found here: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_cluster-planning/bk_cluster-planning.pdf
I would suggest using Ambari to manage your cluster where the memory allocation and settings/config will be much more visible in the UI for each technology/service being used in your cluster.
Created 08-01-2017 02:38 PM
Hi @Hardik Dave
Cluster sizing and planning would require much more detail and in-depth conversation about the use case, the data sizing, etc. A good guide that can help you down the path of sizing your cluster can be found here: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_cluster-planning/bk_cluster-planning.pdf
I would suggest using Ambari to manage your cluster where the memory allocation and settings/config will be much more visible in the UI for each technology/service being used in your cluster.