Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Data visible in hive table but not in impala for data files inside subdirectories

avatar
Explorer

I created an external hive table pointing to directory

 

LOCATION '/user/hive/warehouse/logs'

Inside logs directory I have multiple subfolders as follows:

 

 

/user/hive/warehouse/logs/test/day=20170606
/user/hive/warehouse/logs/test/day=20170607
/user/hive/warehouse/logs/test/day=20170608

I have also used the config:

 

set hive.mapred.supports.subdirectories=true;
set mapred.input.dir.recursive=true;

Using the 'hive' cli I am able to query the data and see the results

 

But when I perform query using Impala I see no results. Do i need to provide some configuration here

 

2 REPLIES 2

avatar
Champion

Because you defined a new the table from outside of impala , you need to refresh hive metadata in impala.

 

fire this command in the impala-shell or Hue

INVALIDATE METADATA

You should be able to query the table

avatar
New Contributor

Reading directories recursively isn't supported by Impala, only Hive. There is a closed JIRA asking for this to be added.

 

https://issues.apache.org/jira/browse/IMPALA-1944#