Is there a way to query streaming data from storm in impala?

[#1]

We want an architecture where streaming data from Storm is landed on to a table folder which can be picked up by Impala. The Impala query will power a dashboard which will be updated every few minutes. Do you have to do regularly "invalidate metadata". But that takes too long. Is there a faster way of doing this?

[#2 · Accepted solution]

Try using "refresh <table name>" instead of "invalidate metadata". This will incrementally update the specified table's metadata rather than clearing all cached metadata, so it should be faster.