Reply
Highlighted
New Contributor
Posts: 2
Registered: ‎03-22-2018

An error occurred retrieving schema from bytes HIVE-HBASE-AVRO

I stored a data with a avro format into a hbase column and I try to mapp the content of this column with a hive table but I had a below error :


My table hive is :

    DROP TABLE IF EXISTS luxdwh.smsc_agg_hbase;
    CREATE EXTERNAL TABLE luxdwh.smsc_agg_hbase (
    )
    ROW FORMAT SERDE 'org.apache.hadoop.hive.hbase.HBaseSerDe'
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES (
        "hbase.columns.mapping" = ":key, smsc_agg:message",
        "smsc_agg.message.serialization.type" = "avro",
        "smsc_agg.message.avro.schema.url" = "schema.avsc")
    TBLPROPERTIES (
        "hbase.table.name" = "smsc_agg",
        "hbase.struct.autogenerate"="true");



>     2018-03-22 09:54:30,368 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl:
> Diagnostics report from attempt_1521646867123_0065_m_000000_2: Error:
> java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
> while processing row [Error getting row data with exception
> org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorException: An
> error occurred retrieving schema from bytes
>         at org.apache.hadoop.hive.serde2.avro.AvroLazyObjectInspector.retrieveSchemaFromBytes(AvroLazyObjectInspector.java:333)
>         at org.apache.hadoop.hive.serde2.avro.AvroLazyObjectInspector.deserializeStruct(AvroLazyObjectInspector.java:289)
>         at org.apache.hadoop.hive.serde2.avro.AvroLazyObjectInspector.getStructFieldData(AvroLazyObjectInspector.java:145)
>         at org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldData(LazySimpleStructObjectInspector.java:117)
>         at org.apache.hadoop.hive.serde2.SerDeUtils.buildJSONString(SerDeUtils.java:353)
>         at org.apache.hadoop.hive.serde2.SerDeUtils.getJSONString(SerDeUtils.java:197)
>         at org.apache.hadoop.hive.serde2.SerDeUtils.getJSONString(SerDeUtils.java:183)
>         at org.apache.hadoop.hive.ql.exec.MapOperator.toErrorMessage(MapOperator.java:529)
>         at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:502)
>         at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:170)
>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
>     Caused by: java.io.IOException: Invalid int encoding
>         at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:145)
>         at org.apache.avro.io.BinaryDecoder.readBytes(BinaryDecoder.java:282)
>         at org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:112)
>         at org.apache.avro.file.DataFileStream.<init>(DataFileStream.java:84)
>         at org.apache.hadoop.hive.serde2.avro.AvroLazyObjectInspector.retrieveSchemaFromBytes(AvroLazyObjectInspector.java:330)
>         ... 17 more

    
       

New Contributor
Posts: 2
Registered: ‎07-02-2018

Re: An error occurred retrieving schema from bytes HIVE-HBASE-AVRO

Hi were you able to resolve this issue? I am facing exact same error and trying to load hbase column data into hive. Any help is appreciated. 

Announcements