I have a number of simple Avro objects stored in HBase and am trying to access them from Hive. I've set up a Hive table by following the instructions that I found here. Basically in Hive I do: DROP TABLE IF EXISTS HBaseAvro; CREATE EXTERNAL TABLE HBaseAvro ROW FORMAT SERDE 'org.apache.hadoop.hive.hbase.HBaseSerDe' STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,event:pCol",
"event.pCol.serialization.type" = "avro",
"event.pCol.avro.schema.url" = "hdfs:///tmp/kafka/avro/avro.avsc") TBLPROPERTIES("hbase.table.name" = "avro",
"hbase.mapred.output.outputtable" = "avro",
"hbase.struct.autogenerate" = "true"); If the Avro object contains the schema in the header, I have no problem and can access the data. However if the Avro object DOes NOT contain the schema then when I try and access the Avro object I get an IO Exception: {"message":"H170 Unable to fetch results. java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating event_pcol... If I do a DESCRIBE on the Hive table, I can see the table correctly, in that event_pcol is shown as a structure, with the correct fields. I've tried moving the avsc file to check that the CREATE TABLE is working and Hive correctly complains. With the CREATE as above the table appears to be created correctly and I can access the "key" values, so the problem appears to be with the Avro object. To me it looks like Hive is not using the schema definition passed in the schema.url parameter. I've tried including the schema as a schema.literal parameter and it still fails. Any ideas?
... View more
