Support Questions

Find answers, ask questions, and share your expertise

Kafka jobs failing : java.lang.AssertionError

Contributor

Our jobs are failing in kafka with error "java.lang.AssertionError: assertion failed: Beginning offset 511 is after the ending offset 510 for topic ps-control partition 6. You either provided an invalid fromOffset, or the Kafka topic has been damaged"

4 REPLIES 4

Contributor

@Geoffrey Shelton Okot

please help here.


New Contributor

I believe this issue was already resolved. In this particular case, the ISR were not updated appropriately and the offsets on couple of brokers were not in sync with rest of them. Once this issue was fixed, the error was no longer visible.

The offsets not being in sync is attributed to network incident where we had continuous flaps for a long duration of time .

Contributor

How it is fixed?

Explorer

Did you find a solution ?

We had the same issue after a kafka cluster reboot. The spark streaming could not start. It could not read in kafka, with the same error. Our environment : HDP 2.6.2/Kafka 0.10.1/Spark Streaming 2.1 Kafka direct with commitAsync

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.