time to provision a spark cluster

asim_munshi — Fri, 16 Sep 2022 10:19:29 GMT

setting up a 15 node cluster; if required consider 3 master nodes and 12 data nodes. setting up HDFS-YARN-SPARK; considering hardware is available and setup with Suse Linux -- how much time should a Project Mgr allocate to provision a production worthy cluster using Blueprints?

Re: time to provision a spark cluster

drussell — Mon, 16 May 2016 15:32:14 GMT

Hi @Asim munshi. This really depends on a lot of variables.

Is this the first time that the team is doing this? Assuming they're using Ambari, that they're exprienced in this and have done it before in several other different environments, but they're also looking to perform performance and stability testing before handover etc I would recommend 1 week which includes at least a day and a half of contingency for things that crop up during deployment.

If however this environment has been used before i.e. networking, firewalling, routing, etc etc is all in a known good state, and the hardware is all 100% correctly provisioned you could easily get everything done including Ambari and the whole cluster in a morning.

I realise that's quite a wide range of times but hopefully that gives you some kind of range to work with.

Hope that helps.

Re: time to provision a spark cluster

asim_munshi — Tue, 17 May 2016 07:57:20 GMT

thanks @drussell

question time to provision a spark cluster in Archives of Support Questions (Read Only)

time to provision a spark cluster

Re: time to provision a spark cluster

Re: time to provision a spark cluster