Reply
PSK
New Contributor
Posts: 14
Registered: ‎12-13-2016

Custom basename for HDFS part files

Hello

 

When we do an INSERT INTO a table in Hive - it creates multiple part files.

 

e.g. part-r-*****   or (in my case) 000000_0, 000001_0 etc.

 

I would like to have custom names for the part files (000000_0 etc). Is it possible to control the basename of these files in Hive query?

 

I have tried the below. But, it is not working (especially when reducer is set to 0).

 

set mapreduce.output.basename='part';

 

Any help/pointers is much appreciaated. Thanks.

Cloudera Employee
Posts: 213
Registered: ‎03-23-2015

Re: Custom basename for HDFS part files

There is no easy way, you might need to create some custom class, see here:

https://wiki.apache.org/hadoop/FAQ#How_do_I_change_final_output_file_name_with_the_desired_name_rath...
PSK
New Contributor
Posts: 14
Registered: ‎12-13-2016

Re: Custom basename for HDFS part files

How is that it differs from part- to 000000_0? I will be happy with part- instead of 00000n_0 format. Any way to change between them?
Announcements