Support Questions
Find answers, ask questions, and share your expertise

Hive Partition

Solved Go to solution

Hive Partition

New Contributor

Hive partition – I have set 10 partition in a table, how can I reduce it to 6 partition without dropping table from HDFS?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Hive Partition

Guru
I don't see super easy way, but below might help:

1. if this table is not external, make it external first:
ALTER TABLE <table> SET TBLPROPERTIES('EXTERNAL'='TRUE')

2. drop the table

3. create a new table and define same partition column

4. manually move files in another 4 partition directories into the 6 partition directories that you want to keep evenly

5. remove the empty directories under the table

6. msck repair table to recover the 6 current partitions

Hope above can help.

Eric

View solution in original post

3 REPLIES 3

Re: Hive Partition

Mentor

You can  use the  below hdfs command

 

$ hdfs dfs -cat hive_table_data_folder/p* > new_file_name

Let me know whether that helped

Re: Hive Partition

Guru
I think the question was about Partitions, not Files?

Re: Hive Partition

Guru
I don't see super easy way, but below might help:

1. if this table is not external, make it external first:
ALTER TABLE <table> SET TBLPROPERTIES('EXTERNAL'='TRUE')

2. drop the table

3. create a new table and define same partition column

4. manually move files in another 4 partition directories into the 6 partition directories that you want to keep evenly

5. remove the empty directories under the table

6. msck repair table to recover the 6 current partitions

Hope above can help.

Eric

View solution in original post