Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Solved
Go to solution
Hive Partition
Labels:
- Labels:
-
Apache Hive
-
HDFS
Explorer
Created on 04-23-2019 10:39 PM - edited 09-16-2022 07:19 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hive partition – I have set 10 partition in a table, how can I reduce it to 6 partition without dropping table from HDFS?
1 ACCEPTED SOLUTION
Super Guru
Created 05-07-2019 09:15 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I don't see super easy way, but below might help:
1. if this table is not external, make it external first:
ALTER TABLE <table> SET TBLPROPERTIES('EXTERNAL'='TRUE')
2. drop the table
3. create a new table and define same partition column
4. manually move files in another 4 partition directories into the 6 partition directories that you want to keep evenly
5. remove the empty directories under the table
6. msck repair table to recover the 6 current partitions
Hope above can help.
Eric
1. if this table is not external, make it external first:
ALTER TABLE <table> SET TBLPROPERTIES('EXTERNAL'='TRUE')
2. drop the table
3. create a new table and define same partition column
4. manually move files in another 4 partition directories into the 6 partition directories that you want to keep evenly
5. remove the empty directories under the table
6. msck repair table to recover the 6 current partitions
Hope above can help.
Eric
3 REPLIES 3
Master Mentor
Created 04-28-2019 09:12 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You can use the below hdfs command
$ hdfs dfs -cat hive_table_data_folder/p* > new_file_name
Let me know whether that helped
Super Guru
Created 05-07-2019 09:12 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think the question was about Partitions, not Files?
Super Guru
Created 05-07-2019 09:15 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I don't see super easy way, but below might help:
1. if this table is not external, make it external first:
ALTER TABLE <table> SET TBLPROPERTIES('EXTERNAL'='TRUE')
2. drop the table
3. create a new table and define same partition column
4. manually move files in another 4 partition directories into the 6 partition directories that you want to keep evenly
5. remove the empty directories under the table
6. msck repair table to recover the 6 current partitions
Hope above can help.
Eric
1. if this table is not external, make it external first:
ALTER TABLE <table> SET TBLPROPERTIES('EXTERNAL'='TRUE')
2. drop the table
3. create a new table and define same partition column
4. manually move files in another 4 partition directories into the 6 partition directories that you want to keep evenly
5. remove the empty directories under the table
6. msck repair table to recover the 6 current partitions
Hope above can help.
Eric
