question Re: java.io.IOException:java.io.EOFException: Read past end of RLE integer from compressed stream in Support Questions

java.io.IOException:java.io.EOFException: Read past end of RLE integer from compressed stream

james_m_barney — Mon, 14 Dec 2015 23:25:49 GMT

Trying to run a simple select column statement on a hive table stored as ORC and I'm running into this error. I can't seem to find any solution to it, however, I have found similar questions around the internet regarding it.

Failed with exception java.io.IOException:java.io.EOFException: Read past end of RLE integer from compressed stream Stream for column 2 kind DATA position: 20 length: 20 range: 0 offset: 20 limit: 20 range 0 = 0 to 20 uncompressed: 77 to 77

The data looks like this (here it's pseudo-data, tab-separated):

uuid	feed	unixgmtfetchtime
aaa	reviews	1385830800
bbb	reviews	1395830800
zzz	reviews	1405830800

with schema:

uuid                	string
feed                	string
unixgmtfetchtime        int

running select unixgmtfetchtime from tablename; returns the error mentioned above.

More info: I have run alter table tablename concatenate; on the table.

Re: java.io.IOException:java.io.EOFException: Read past end of RLE integer from compressed stream

aervits — Thu, 24 Dec 2015 22:25:12 GMT

@James Barney did you try using string or one of the date types for unixgmtfetchtime?

Re: java.io.IOException:java.io.EOFException: Read past end of RLE integer from compressed stream

aervits — Wed, 03 Feb 2016 10:41:00 GMT

@James Barney has this been resolved? Can you post your solution or accept the best answer?

Re: java.io.IOException:java.io.EOFException: Read past end of RLE integer from compressed stream

james_m_barney — Wed, 03 Feb 2016 22:04:04 GMT

This was never resolved by any action on my part, it simply started working again and I'm not sure why. However: some more information--

@Artem Ervits I can't use date or string due to what the column actually is used for.

Additionally, I was able to read up to a certain point in the table by including 'limit xx' in my query. After a certain amount of rows, the error would be thrown. I believe it had something to do with a corrupted row in the table's files. I don't know how it got there or how it fixed itself, but after having the problem for many days, it resolved itself.

Re: java.io.IOException:java.io.EOFException: Read past end of RLE integer from compressed stream

sasampangiraman — Fri, 03 Jun 2016 02:52:41 GMT

I am currently having exact same problem as well and curious to know if any others seen this issue (and a possible resoluton).

Re: java.io.IOException:java.io.EOFException: Read past end of RLE integer from compressed stream

sushmatelukunta — Thu, 18 Aug 2016 00:22:07 GMT

I am currently having exact same problem as well and curious to know if any others seen this issue (and a possible resoluton).

Also noticed that sometimes this issue comes after merging small files. And like mentioned before its random, doesn't happen on all partitions in my table.

Any solution would be good.

I am currently having exact same problem as well and curious to know if any others seen this issue (and a possible resoluton).

I also noticed that this comes some times after merging small files.

Any solutions would be good.