https://community.cloudera.com/t5/Archives-of-Support-Questions/How-can-I-control-the-size-of-the-blocks-that-hive-writes-to/m-p/161103#M45110 <P>@Peter Coates - look for the parameters <CODE>fs.s3a.multipart.threshold</CODE> and <CODE>fs.s3a.multipart.size</CODE></P> Wed, 02 Nov 2016 21:00:17 GMT njayakumar 2016-11-02T21:00:17Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-can-I-control-the-size-of-the-blocks-that-hive-writes-to/m-p/161102#M45109 <P>I'm using insert-into to write data up to S3, but it's writing very large files--0.8GB to 1.8 GB plus one of just a few K. I've tried tez.grouping.max-size and min-size, but neither seems to limit either the min or the max size of the files that are generated. I've also tried controlling the number of mappers and reducers, but to no avail. </P> Wed, 02 Nov 2016 20:55:23 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-can-I-control-the-size-of-the-blocks-that-hive-writes-to/m-p/161102#M45109 coatespt 2016-11-02T20:55:23Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-can-I-control-the-size-of-the-blocks-that-hive-writes-to/m-p/161103#M45110 <P>@Peter Coates - look for the parameters <CODE>fs.s3a.multipart.threshold</CODE> and <CODE>fs.s3a.multipart.size</CODE></P> Wed, 02 Nov 2016 21:00:17 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-can-I-control-the-size-of-the-blocks-that-hive-writes-to/m-p/161103#M45110 njayakumar 2016-11-02T21:00:17Z