Support Questions

Find answers, ask questions, and share your expertise

Data visible in hive table but not in impala for data files inside subdirectories

avatar
Explorer

I created an external hive table pointing to directory

 

LOCATION '/user/hive/warehouse/logs'

Inside logs directory I have multiple subfolders as follows:

 

 

/user/hive/warehouse/logs/test/day=20170606
/user/hive/warehouse/logs/test/day=20170607
/user/hive/warehouse/logs/test/day=20170608

I have also used the config:

 

set hive.mapred.supports.subdirectories=true;
set mapred.input.dir.recursive=true;

Using the 'hive' cli I am able to query the data and see the results

 

But when I perform query using Impala I see no results. Do i need to provide some configuration here

 

2 REPLIES 2

avatar
Champion

Because you defined a new the table from outside of impala , you need to refresh hive metadata in impala.

 

fire this command in the impala-shell or Hue

INVALIDATE METADATA

You should be able to query the table

avatar
New Contributor

Reading directories recursively isn't supported by Impala, only Hive. There is a closed JIRA asking for this to be added.

 

https://issues.apache.org/jira/browse/IMPALA-1944#