There are pros and cons of both approaches. VM based pros: 1. 'Easier' managing nodes. Some IT infrastructure teams insist on VMs even if you want to map 1 physical node to 1 virtual node because all their other infrastructure is based on VMs. 2. Taking advantage of NUMA and memory locality. There are some articles on this from virtual infrastructure providers that you can take a look at. VM based disadvantages: 1. Overhead. As an example, if you are running 4VMs per physical node, you are running 4 OS, 4 Datanode services, 4 Nodemanagers, 4 ambari-agents, 4 metrics collectors and 4 of any other worker services instead of one. These services will have overhead compared to running 1 of each. 2. Data Locality and redundancy. Now, there is support to know physical nodes, so no two replicas go into same physical node but that is extra configuration. You might run into virtual disk performance problems if they are not configured properly. Given a choice, I prefer using Physical servers. However, its not always your choice. In those cases, make sure you try to get following. 1. Explicit virtual disk to physical disk mapping. Say you have 2 VMs per physical node and each physical node has 16 data drives. Make sure to split 8 drives to one VM and 8 more to another VM. This way, physical disks are not shared between VMs. 2. Don't go for more than 2 VMs per physical node. This is so you minimize overhead from the services running. Regarding your question of mixing physical and virtual machines, try to see that all your worker nodes are of similar hardware. While heterogenous hardware is supported, you can run into issues because nodes have different hardware profiles. However, we had some customers who used VMs for master services and physical nodes for worker nodes. This was one way to getting away from NN SPOF issues in Hadoop1 days.
... View more
