Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Kafka jobs failing : java.lang.AssertionError

avatar
Rising Star

Our jobs are failing in kafka with error "java.lang.AssertionError: assertion failed: Beginning offset 511 is after the ending offset 510 for topic ps-control partition 6. You either provided an invalid fromOffset, or the Kafka topic has been damaged"

4 REPLIES 4

avatar
Rising Star

@Geoffrey Shelton Okot

please help here.


avatar
New Contributor

I believe this issue was already resolved. In this particular case, the ISR were not updated appropriately and the offsets on couple of brokers were not in sync with rest of them. Once this issue was fixed, the error was no longer visible.

The offsets not being in sync is attributed to network incident where we had continuous flaps for a long duration of time .

avatar
Rising Star

How it is fixed?

avatar
Explorer

Did you find a solution ?

We had the same issue after a kafka cluster reboot. The spark streaming could not start. It could not read in kafka, with the same error. Our environment : HDP 2.6.2/Kafka 0.10.1/Spark Streaming 2.1 Kafka direct with commitAsync