Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

In some orc tables in Hive we get duplicate partition "base" directory inside /table_name/partition_date=/base/

Highlighted

In some orc tables in Hive we get duplicate partition "base" directory inside /table_name/partition_date=/base/

Contributor

Hi All,

Periodically, in some ORC tables in Hive we get duplicate partition "base" directory inside /table_name/partition_date=/base/ meaning: all contents of /table_name/partition_date=/base/* are in /table_name/partition_date=/base/base/*. After that partition become bad and from this bad partition we can’t do select count(*) or any other selects because of error occurring.

But when we dropping duplicate “base” directory problem goes away.

Why we got this duplicate folder in our buckets?

2 REPLIES 2

Re: In some orc tables in Hive we get duplicate partition "base" directory inside /table_name/partition_date=/base/

Expert Contributor

Could you give the exact table DDL and "ls -R" directory listing of the partition?

Re: In some orc tables in Hive we get duplicate partition "base" directory inside /table_name/partition_date=/base/

Contributor

This event occurs only if we are using NiFi Hive Streaming.

ls -R will later.

Don't have an account?
Coming from Hortonworks? Activate your account here