Support Questions

Find answers, ask questions, and share your expertise

Hive Partition

avatar
Explorer

Hive partition – I have set 10 partition in a table, how can I reduce it to 6 partition without dropping table from HDFS?

1 ACCEPTED SOLUTION

avatar
Super Guru
I don't see super easy way, but below might help:

1. if this table is not external, make it external first:
ALTER TABLE <table> SET TBLPROPERTIES('EXTERNAL'='TRUE')

2. drop the table

3. create a new table and define same partition column

4. manually move files in another 4 partition directories into the 6 partition directories that you want to keep evenly

5. remove the empty directories under the table

6. msck repair table to recover the 6 current partitions

Hope above can help.

Eric

View solution in original post

3 REPLIES 3

avatar
Master Mentor

You can  use the  below hdfs command

 

$ hdfs dfs -cat hive_table_data_folder/p* > new_file_name

Let me know whether that helped

avatar
Super Guru
I think the question was about Partitions, not Files?

avatar
Super Guru
I don't see super easy way, but below might help:

1. if this table is not external, make it external first:
ALTER TABLE <table> SET TBLPROPERTIES('EXTERNAL'='TRUE')

2. drop the table

3. create a new table and define same partition column

4. manually move files in another 4 partition directories into the 6 partition directories that you want to keep evenly

5. remove the empty directories under the table

6. msck repair table to recover the 6 current partitions

Hope above can help.

Eric