Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Impala: Failed to parse view-definition statement (HBase wide column -> Hive -> Impala)

Impala: Failed to parse view-definition statement (HBase wide column -> Hive -> Impala)

New Contributor

Dear all,

 

I am having troubles using Impala to access data stored in HBase.

 

Impala throws me the following error message:

 

AnalysisException: Failed to load metadata for table: default.customsink_lat CAUSED BY: TableLoadingException: TableLoadingException: Failed to parse view-definition statement of view: default.customsink_lat

However, Hive does work perfectly fine. Unfortunately it is not an option because it is too slow. 

 

The scenario:

- One HBase table with wide columns

- One Hive table based on the HBase table and another one that uses explode to transform the columns into rows

 

The structure in Hive:

 

customsink_lat:

hive structure.png

based on this code:

 

create or replace view customsink_lat as
SELECT id, clmndate,clmnvalue FROM customsink  LATERAL VIEW explode(props) cl AS clmndate,clmnvalue;

 and another one based directly on HBase:

CREATE EXTERNAL TABLE customsink(
id string,
props map<bigint,float>
)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
  "hbase.columns.mapping" =
  ":key,cf:"
)
TBLPROPERTIES("hbase.table.name" = "customsink");

 

Do you have any idea why Imala cannot handle the structure of customsink_lat?

 

Thank you very much in advance,

Fabian

 

PS: Here are the results of Hive queries.

Especially customsink_lat does not contain nested data anymore, although it is based on nested data. So I cannot find an explanation why Impala still refuses to parse customsink_lat.

 

hive> select * from customsink;                              
OK
Source1	{9223370642321980337:1.0,9223370642321980339:0.0,9223370642329180337:1.0,9223370642329180339:0.0}
Source2	{9223370642321980333:3.0,9223370642321980335:2.0,9223370642329180333:3.0,9223370642329180335:2.0}
Time taken: 0.296 seconds

 

hive> select * from customsink_lat;
Total MapReduce jobs = 1
Launching Job 1 out of 1
OK
Source1	9223370642321980337	1.0
Source1	9223370642321980339	0.0
Source1	9223370642329180337	1.0
Source1	9223370642329180339	0.0
Source2	9223370642321980333	3.0
Source2	9223370642321980335	2.0
Source2	9223370642329180333	3.0
Source2	9223370642329180335	2.0
Time taken: 27.986 seconds

 

2 REPLIES 2
Highlighted

Re: Impala: Failed to parse view-definition statement (HBase wide column -> Hive -> Impala)

Master Collaborator

Hive and Impala only support non-materialized VIEWs that don't contain any actual data.

So when you write a query

 

select * from customsink_lat

 

both Hive and Impala replace 'customsink_lat' with the view-definition SQL like this:

 

select v.* from

  (SELECT id, clmndate,clmnvalue FROM customsink LATERAL VIEW explode(props) cl AS clmndate,clmnvalue) v

 

Impala doesn't support LATERAL VIEW or complex types such as MAP, ARRAY, etc., so Impala isn't able to execute the view-definition SQL.

 

My apologies for the inconvenience.

Re: Impala: Failed to parse view-definition statement (HBase wide column -> Hive -> Impala)

New Contributor

Is there any work around for this issue?