Explorer
Posts: 9
Registered: ‎10-28-2015
combine small parquet files

hi all,

 

i have a problem. I have one table in Impala, partition by year, month, day, name of the load file.  Example

 

 

/user/hive/warehouse/table/year=2015/month=01/day=01/nameloadfile1/file.parq

/user/hive/warehouse/table/year=2015/month=01/day=01/nameloadfile1/file2.parq

/user/hive/warehouse/table/year=2015/month=01/day=01/nameloadfile1/file3.parq

 

the size of the file is very smaller (low MB). I would want combine all files in one.

 

/user/hive/warehouse/table/year=2015/month=01/day=01/nameloadfile1/combineFiles.parq

 

Someone have any idea for that.

 

Best regards

 

 

Who Me Too'd this topic