I am trying to split file size to 64mb

[#1]

set mapred.max.split.size=67108864; set mapred.min.split.size=1024; set hive.execution.engine=tez; insert overwrite table bdd.signal_hte partition(cvdt36_year,cvdt36_mon,cvdt36_day) select * from cv.signal_hte where cvdt36_year= "2015" and cvdt36_mon =05;

[#2 · Accepted solution]

@Akhil Reddy

For tez, you need to use below parameter to set min and max splits of data:

1. set tez.grouping.min-size=16777216;--16 MB min split
2. set tez.grouping.max-size=64000000;--64 GB max split

Increase min and max split size to reduce the number of mappers.

[#3]

1003608529

still the size remain same

[#4]

if I do it for one day record it is working If I do for entire one year it still remain same.

[#5]

insert overwrite table Mynewtable partition(cvdt36_year,cvdt36_mon,cvdt36_day) select * from MainTable where cvdt36_year= "2015" and cvdt36_mon =05 and cvdt36_day=16;

If I run this query it is working fine

insert overwrite table Mynewtable partition(cvdt36_year,cvdt36_mon,cvdt36_day) select * from MainTable where cvdt36_year= 2015; it is not working can you suggest me with correct query.

[#6]

@Akhil Reddy

There is no syntax issue with query, could you please share the issue you are facing?