I am trying to plan a kafka cluster on AWS. I am wondering if anyone can tell me what should I consider for the CPU and memory sizing. Is there a link which can provide good details of proper kafka node size planning. To give you a bit of background, I am planning to use three gateway nodes in which kafka and streamset will be used to ingest the data (kafka installed in three nodes and streamset installed in one node). Is this a good setup or I should install both kafka and streamset on all three nodes? Look forward to hearing from you.
There are lots of factors that go into sizing a Kafka cluster, but you can high level get a ballpark based on throughput and storage requirements. See link for more info https://medium.com/oracledevs/sizingeventhubcluster-dbb639e42094