Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka jobs failing : java.lang.AssertionError

Highlighted

Kafka jobs failing : java.lang.AssertionError

Contributor

Our jobs are failing in kafka with error "java.lang.AssertionError: assertion failed: Beginning offset 511 is after the ending offset 510 for topic ps-control partition 6. You either provided an invalid fromOffset, or the Kafka topic has been damaged"

4 REPLIES 4

Re: Kafka jobs failing : java.lang.AssertionError

Contributor

@Geoffrey Shelton Okot

please help here.


Re: Kafka jobs failing : java.lang.AssertionError

New Contributor

I believe this issue was already resolved. In this particular case, the ISR were not updated appropriately and the offsets on couple of brokers were not in sync with rest of them. Once this issue was fixed, the error was no longer visible.

The offsets not being in sync is attributed to network incident where we had continuous flaps for a long duration of time .

Re: Kafka jobs failing : java.lang.AssertionError

Contributor

How it is fixed?

Re: Kafka jobs failing : java.lang.AssertionError

New Contributor

Did you find a solution ?

We had the same issue after a kafka cluster reboot. The spark streaming could not start. It could not read in kafka, with the same error. Our environment : HDP 2.6.2/Kafka 0.10.1/Spark Streaming 2.1 Kafka direct with commitAsync