Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Impala: Failed to parse view-definition statement (HBase wide column -> Hive -> Impala)

Impala: Failed to parse view-definition statement (HBase wide column -> Hive -> Impala)

New Contributor

Dear all,

 

I am having troubles using Impala to access data stored in HBase.

 

Impala throws me the following error message:

 

AnalysisException: Failed to load metadata for table: default.customsink_lat CAUSED BY: TableLoadingException: TableLoadingException: Failed to parse view-definition statement of view: default.customsink_lat

However, Hive does work perfectly fine. Unfortunately it is not an option because it is too slow. 

 

The scenario:

- One HBase table with wide columns

- One Hive table based on the HBase table and another one that uses explode to transform the columns into rows

 

The structure in Hive:

 

customsink_lat:

hive structure.png

based on this code:

 

create or replace view customsink_lat as
SELECT id, clmndate,clmnvalue FROM customsink  LATERAL VIEW explode(props) cl AS clmndate,clmnvalue;

 and another one based directly on HBase:

CREATE EXTERNAL TABLE customsink(
id string,
props map<bigint,float>
)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
  "hbase.columns.mapping" =
  ":key,cf:"
)
TBLPROPERTIES("hbase.table.name" = "customsink");

 

Do you have any idea why Imala cannot handle the structure of customsink_lat?

 

Thank you very much in advance,

Fabian

 

PS: Here are the results of Hive queries.

Especially customsink_lat does not contain nested data anymore, although it is based on nested data. So I cannot find an explanation why Impala still refuses to parse customsink_lat.

 

hive> select * from customsink;                              
OK
Source1	{9223370642321980337:1.0,9223370642321980339:0.0,9223370642329180337:1.0,9223370642329180339:0.0}
Source2	{9223370642321980333:3.0,9223370642321980335:2.0,9223370642329180333:3.0,9223370642329180335:2.0}
Time taken: 0.296 seconds

 

hive> select * from customsink_lat;
Total MapReduce jobs = 1
Launching Job 1 out of 1
OK
Source1	9223370642321980337	1.0
Source1	9223370642321980339	0.0
Source1	9223370642329180337	1.0
Source1	9223370642329180339	0.0
Source2	9223370642321980333	3.0
Source2	9223370642321980335	2.0
Source2	9223370642329180333	3.0
Source2	9223370642329180335	2.0
Time taken: 27.986 seconds

 

2 REPLIES 2

Re: Impala: Failed to parse view-definition statement (HBase wide column -> Hive -> Impala)

Master Collaborator

Hive and Impala only support non-materialized VIEWs that don't contain any actual data.

So when you write a query

 

select * from customsink_lat

 

both Hive and Impala replace 'customsink_lat' with the view-definition SQL like this:

 

select v.* from

  (SELECT id, clmndate,clmnvalue FROM customsink LATERAL VIEW explode(props) cl AS clmndate,clmnvalue) v

 

Impala doesn't support LATERAL VIEW or complex types such as MAP, ARRAY, etc., so Impala isn't able to execute the view-definition SQL.

 

My apologies for the inconvenience.

Highlighted

Re: Impala: Failed to parse view-definition statement (HBase wide column -> Hive -> Impala)

New Contributor

Is there any work around for this issue?