Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Does Alter table Concatenate cause data issues such as duplicates

Highlighted

Does Alter table Concatenate cause data issues such as duplicates

New Contributor

To address small files performance issue we have been concatenating small files by running this statement in Hive.

ALTER TABLE ${database}.${table1} PARTITION(policy_symbol="${policy_symbol}",transaction_year_month="${transaction_year_month}") CONCATENATE

We have since discovered duplicate data and are wondering if this is due to a bug in the concatenate command?

Has anyone else encountered similar?

Thanks.

Andy