Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Offset issue when changing zookeeper root path in spark

Highlighted

Offset issue when changing zookeeper root path in spark

Contributor

Hi all,


Witnessing some unexpected behavior i'm hoping someone might be able to clarify for me.

I have a spark job which we set the zookeeper root to "/example" and the consumer group to "example_group", the offset here is

2:101

1:104

0:106

Using the same consumer group i switched the zookeeper root to "/example_temp" the idea here being that we can continue testing our job on "/example_temp" while the messages are still sent to the kafka topic and when we switch back to "/example" we'll ingest all the messages since the last offset for "/example", allowing us to test a larger load. Now the weird bit is that the offset in "/example_temp" is lower than that of "/example"

2:65

1:67

0:66

And so when I switched back to "/example", i got this error

java.lang.IllegalArgumentException: requirement failed: numRecords must not be negative

Because the offset saved on the broker is that of "/example_temp" and this is lower than the value saved in zookeeper for "/example"

if you're still with me, my question is how did the offset for "/example_temp" end up lower than that of "/example"?