question Minimum number of nodes, and specs for a real cluster in Support Questions

question Minimum number of nodes, and specs for a real cluster in Support Questions https://community.cloudera.com/t5/Support-Questions/Minimum-number-of-nodes-and-specs-for-a-real-cluster/m-p/33096#M52140 Hi   I've been tasked with setting up a Hadoop cluster for testing a new big data initiative. However I'm pretty much completely new to all of this. I know that one can set up a single node cluster for proof of concept, but I would like to know what is the minimum number of nodes, and what spec (amount of RAM & disk space) for a proper cluster. Imagine a low throughput as it's only an initial test cluster (fewer than 10 users). And we only need Kafka, HDFS, Pig & Hive services to run.   We generally have the ability to spin up Centos 6 VM's with 4GB RAM each, and I might be able to up that to 8GB each. But Reading many of the setup pages, it's quoting minimums of 10s of GB of RAM (e.g. <A href="http://blog.cloudera.com/blog/2013/08/how-to-select-the-right-hardware-for-your-new-hadoop-cluster/)," target="_blank">http://blog.cloudera.com/blog/2013/08/how-to-select-the-right-hardware-for-your-new-hadoop-cluster/),</A> but the cloudera manager setup only asks for at least 4GB on that node (<A href="http://www.cloudera.com/content/www/en-us/documentation/enterprise/5-3-x/topics/cm_ig_cm_requirements.html)" target="_blank">http://www.cloudera.com/content/www/en-us/documentation/enterprise/5-3-x/topics/cm_ig_cm_requirements.html)</A> and mentions nothing around the other node's specs.   Let me know if you need any more information. I realise it's probably too vague as is.   Cheers, Ed Fri, 16 Sep 2022 09:44:40 GMT edwardmlyte 2022-09-16T09:44:40Z