Support Questions

Find answers, ask questions, and share your expertise
Celebrating as our community reaches 100,000 members! Thank you!

Impala: Failed to parse view-definition statement (HBase wide column -> Hive -> Impala)


Dear all,


I am having troubles using Impala to access data stored in HBase.


Impala throws me the following error message:


AnalysisException: Failed to load metadata for table: default.customsink_lat CAUSED BY: TableLoadingException: TableLoadingException: Failed to parse view-definition statement of view: default.customsink_lat

However, Hive does work perfectly fine. Unfortunately it is not an option because it is too slow. 


The scenario:

- One HBase table with wide columns

- One Hive table based on the HBase table and another one that uses explode to transform the columns into rows


The structure in Hive:



hive structure.png

based on this code:


create or replace view customsink_lat as
SELECT id, clmndate,clmnvalue FROM customsink  LATERAL VIEW explode(props) cl AS clmndate,clmnvalue;

 and another one based directly on HBase:

id string,
props map<bigint,float>
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
  "hbase.columns.mapping" =
TBLPROPERTIES("" = "customsink");


Do you have any idea why Imala cannot handle the structure of customsink_lat?


Thank you very much in advance,



PS: Here are the results of Hive queries.

Especially customsink_lat does not contain nested data anymore, although it is based on nested data. So I cannot find an explanation why Impala still refuses to parse customsink_lat.


hive> select * from customsink;                              
Source1	{9223370642321980337:1.0,9223370642321980339:0.0,9223370642329180337:1.0,9223370642329180339:0.0}
Source2	{9223370642321980333:3.0,9223370642321980335:2.0,9223370642329180333:3.0,9223370642329180335:2.0}
Time taken: 0.296 seconds


hive> select * from customsink_lat;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Source1	9223370642321980337	1.0
Source1	9223370642321980339	0.0
Source1	9223370642329180337	1.0
Source1	9223370642329180339	0.0
Source2	9223370642321980333	3.0
Source2	9223370642321980335	2.0
Source2	9223370642329180333	3.0
Source2	9223370642329180335	2.0
Time taken: 27.986 seconds




Hive and Impala only support non-materialized VIEWs that don't contain any actual data.

So when you write a query


select * from customsink_lat


both Hive and Impala replace 'customsink_lat' with the view-definition SQL like this:


select v.* from

  (SELECT id, clmndate,clmnvalue FROM customsink LATERAL VIEW explode(props) cl AS clmndate,clmnvalue) v


Impala doesn't support LATERAL VIEW or complex types such as MAP, ARRAY, etc., so Impala isn't able to execute the view-definition SQL.


My apologies for the inconvenience.

New Contributor

Is there any work around for this issue?