After some experimenting, here's what we found. I'd be interested if anyone enthusiastically agrees or passionately disagrees this. Tested the following scenarios: 1) 128mb block, 128mb file, gzip 2) 128mb block, 1gb file, gzip 3) 1gb block, 1gb file, gzip 4) 128mb block, 128 file, snappy 5) 128mb block, 1gb file, snappy 6) 1gb block, 1gb file, snappy The worst in storage and performance seemed to be the 2 cases where the block size was much smaller than the file size in both compression formats, so strike out #2 and #5. The performance for 1, 3, 4, and 6 all seemed to be very similar in the queries we tested. But gzip used only about 60% as much storage. So, probably going with gzip. Finally we're thinking the smaller block and file size is probably the way to go to get a little more parallelism.
... View more
