Twitter data ingestion to Hive giving error as follow.
Failed with exception java.io.IOException:org.apache.avro.AvroRuntimeException: java.io.IOException: Block size invalid or too large for this implementation: -40
Please assist me on this.
Thanks in advance.
For example if i type select * from table; it is throwing the above error.
Try to reproduce it as mentioned here -
Alternate approach is to use another flume source to save data in raw json instead.
I was wondering, how did you know the data got loaded in the table without the "select * from" query returning anything? I suggest to
1.create your hive table with proper partitioning or
2. Re-import the avro formated data using flume and make sure the below parameter is set to
Heres a detailed explanation of how to's in hive with avro: