time to provision a spark cluster

[#1]

setting up a 15 node cluster; if required consider 3 master nodes and 12 data nodes. setting up HDFS-YARN-SPARK; considering hardware is available and setup with Suse Linux -- how much time should a Project Mgr allocate to provision a production worthy cluster using Blueprints?

[#2 · Accepted solution]

Hi @Asim munshi. This really depends on a lot of variables.

Is this the first time that the team is doing this? Assuming they're using Ambari, that they're exprienced in this and have done it before in several other different environments, but they're also looking to perform performance and stability testing before handover etc I would recommend 1 week which includes at least a day and a half of contingency for things that crop up during deployment.

If however this environment has been used before i.e. networking, firewalling, routing, etc etc is all in a known good state, and the hardware is all 100% correctly provisioned you could easily get everything done including Ambari and the whole cluster in a morning.

I realise that's quite a wide range of times but hopefully that gives you some kind of range to work with.

Hope that helps.

[#3]

thanks @drussell