Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Data visible in hive table but not in impala for data files inside subdirectories

Data visible in hive table but not in impala for data files inside subdirectories

Explorer

I created an external hive table pointing to directory

 

LOCATION '/user/hive/warehouse/logs'

Inside logs directory I have multiple subfolders as follows:

 

 

/user/hive/warehouse/logs/test/day=20170606
/user/hive/warehouse/logs/test/day=20170607
/user/hive/warehouse/logs/test/day=20170608

I have also used the config:

 

set hive.mapred.supports.subdirectories=true;
set mapred.input.dir.recursive=true;

Using the 'hive' cli I am able to query the data and see the results

 

But when I perform query using Impala I see no results. Do i need to provide some configuration here

 

2 REPLIES 2

Re: Data visible in hive table but not in impala for data files inside subdirectories

Champion

Because you defined a new the table from outside of impala , you need to refresh hive metadata in impala.

 

fire this command in the impala-shell or Hue

INVALIDATE METADATA

You should be able to query the table

Highlighted

Re: Data visible in hive table but not in impala for data files inside subdirectories

New Contributor

Reading directories recursively isn't supported by Impala, only Hive. There is a closed JIRA asking for this to be added.

 

https://issues.apache.org/jira/browse/IMPALA-1944#