Periodically, in some ORC tables in Hive we get duplicate partition "base" directory inside /table_name/partition_date=/base/ meaning: all contents of /table_name/partition_date=/base/* are in /table_name/partition_date=/base/base/*. After that partition become bad and from this bad partition we can’t do select count(*) or any other selects because of error occurring.
But when we dropping duplicate “base” directory problem goes away.
Why we got this duplicate folder in our buckets?
Could you give the exact table DDL and "ls -R" directory listing of the partition?
This event occurs only if we are using NiFi Hive Streaming.
ls -R will later.