Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka jobs failing : java.lang.AssertionError

Kafka jobs failing : java.lang.AssertionError

Contributor

Our jobs are failing in kafka with error "java.lang.AssertionError: assertion failed: Beginning offset 511 is after the ending offset 510 for topic ps-control partition 6. You either provided an invalid fromOffset, or the Kafka topic has been damaged"

4 REPLIES 4
Highlighted

Re: Kafka jobs failing : java.lang.AssertionError

Contributor

@Geoffrey Shelton Okot

please help here.


Highlighted

Re: Kafka jobs failing : java.lang.AssertionError

New Contributor

I believe this issue was already resolved. In this particular case, the ISR were not updated appropriately and the offsets on couple of brokers were not in sync with rest of them. Once this issue was fixed, the error was no longer visible.

The offsets not being in sync is attributed to network incident where we had continuous flaps for a long duration of time .

Highlighted

Re: Kafka jobs failing : java.lang.AssertionError

Contributor

How it is fixed?

Highlighted

Re: Kafka jobs failing : java.lang.AssertionError

Explorer

Did you find a solution ?

We had the same issue after a kafka cluster reboot. The spark streaming could not start. It could not read in kafka, with the same error. Our environment : HDP 2.6.2/Kafka 0.10.1/Spark Streaming 2.1 Kafka direct with commitAsync

Don't have an account?
Coming from Hortonworks? Activate your account here