Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Super slow when describing a Hive partitioned table

avatar
Explorer

Env:
HDP 3.1.5
Hive: hive_3_1_5_0_152
We have a hive table with 6k partitions, it is very normal when describing a single partition but super slow when describing the whole table, describing non-partitioned tables is normal.

desc hive_desc_partition_tbl_01; -- cost 1k+ sec
desc hive_desc_partition_tbl_02 partition(ns='NS1',slicetime=20200102);  -- cost 3 sec
alter table hive_desc_partition_tbl_01 change column id id string comment 'change';  -- cost 3600 sec

 

The DDL of the table:
create TABLE `hive_desc_partition_tbl_01` (
`id` bigint,
`b` string,
`c` string,
`d` string,
`e` string,
`ns_name` string)
COMMENT 'compress test'
PARTITIONED BY (
`ns` string,
`slicetime` string)

 

Any ideas to optimize? Thanks in advance!

1 REPLY 1

avatar
Expert Contributor

I suspect the cause for "alter table" slowness is same as described here

HIVE-23959 & HIVE-23806 might help your situation.