@Ancil McBarnett just wondering, are these specs not going against the fundamental design principles of scaling out? The specifications seems to be very high for me. I thought distributed applications should work well on commodity and cheap hardware specifications. I was of the view that cluster of machines with 8GB, 1TB, 4CPU will do a good job. However this was not the case after I set up a 8 node cluster in Azure, ran a job on 1TB of data. It took 8 hours.
I posted a question about this today and I did tag you.
... View more
