@TheBroMeister I will try to comment my views inline - 1.) How different would the Setup and configuration be for Physical Servers as to VMs. Yes, Setting up the VMs would be faster as compared to the physical ones but are there any additional configurations or settings that we would need to look into? -- If we talk regarding general configuration they below points will be taken in account which counting on performance - a. Disks b Network c. Memory/CPU d. SLA 2.) We've read that one possible issue with setting the cluster on VMs is with Data Locality and redundancy. On how no 2 replicas should not be in the same physical node but since one physical node may house several VMs, would there be a way around this issue? -- VM with external storage[like SAN] will be impacting data locality. You can go with dedicated disk for the VM's which will be a good hybrid approach. 'YES' , also for data locality addon components from virtual vendors[like vmware] are provided - such as BDE [Big Data Extensions] also for Network compromises of NSX technology which will help to speed up systems to avoid performance impacts. But you need to take licensing cost into account. 3.) Since the specs of the VMs would be restricted to the specs of the physical node and its resources be split depending on how many VMs it is housing, wouldn't it be better to have separate servers to house 1 node of a cluster to get better performance? and would having several VMs in one physical node affect the parallelism of the jobs that will run on the cluster? -- Its difficult to put decision at first moment based upon actual experiences. This decision purely depends upon your sla. At start while running hadoop applications, you might not be aware of how much time it takes for your application to process or meet the SLA. This can be purely POC base approach you need to test and also run benchmarking before you go for actual dev/uat/prod implementations. benchmarking results will give you fair idea about performance and computational stats. That can be easy then to take the decision. Pls do check below links which might be useful - https://community.cloudera.com/t5/Support-Questions/Virtual-Machines-in-Hadoop-cluster/td-p/119675 https://www.kdnuggets.com/2015/12/myths-virtualizing-hadoop-vsphere-explained.html https://pubs.vmware.com/bde-2/index.jsp
... View more
