Reply
Explorer
Posts: 6
Registered: ‎06-05-2015

Data visible in hive table but not in impala for data files inside subdirectories

I created an external hive table pointing to directory

 

LOCATION '/user/hive/warehouse/logs'

Inside logs directory I have multiple subfolders as follows:

 

 

/user/hive/warehouse/logs/test/day=20170606
/user/hive/warehouse/logs/test/day=20170607
/user/hive/warehouse/logs/test/day=20170608

I have also used the config:

 

set hive.mapred.supports.subdirectories=true;
set mapred.input.dir.recursive=true;

Using the 'hive' cli I am able to query the data and see the results

 

But when I perform query using Impala I see no results. Do i need to provide some configuration here

 

Highlighted
Champion
Posts: 295
Registered: ‎05-16-2016

Re: Data visible in hive table but not in impala for data files inside subdirectories

[ Edited ]

Because you defined a new the table from outside of impala , you need to refresh hive metadata in impala.

 

fire this command in the impala-shell or Hue

INVALIDATE METADATA

You should be able to query the table

Announcements