Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to improve performance on a table/partition after running concatenate?

Highlighted

How to improve performance on a table/partition after running concatenate?

After running "concatenate" on a partition in hive I can see the no. of files drastically reduced. However when I run a simple select query with limit function it takes longer to display the result. My question: Is this a normal behaviour after concatenate or the query should execute faster after concatenate. Can someone clarify?

1 REPLY 1

Re: How to improve performance on a table/partition after running concatenate?

Expert Contributor

@knarendran to observe performance improvement with partitions, you must have a "WHERE" Clause on partition column, that would allow hive to perform desired partition scan of the table included in the "where" clause and skip non-desired partitions.