question Re: Ambari, set "auto.offset.reset" property for kafka? in Support Questions

Ambari, set "auto.offset.reset" property for kafka?

vinti_uiet — Wed, 02 Mar 2016 04:52:43 GMT

Hi,

I wanted to set "auto.offset.reset" to smallest in kafka consumer, as i my kafka consumer (spark-scala program) is not reading data from beginning currently.

https://kafka.apache.org/08/configuration.html

* smallest : automatically reset the offset to the smallest offset

Thanks & Regards,

Vinti

Re: Ambari, set "auto.offset.reset" property for kafka?

aervits — Wed, 02 Mar 2016 04:55:14 GMT

@Vinti Maheshwari go to kafka configuration page and add the custom property if it's not there or change the value for the property.

Re: Ambari, set "auto.offset.reset" property for kafka?

ammills01 — Wed, 07 Nov 2018 06:14:03 GMT

I realize this is an old answer but it is wrong and I don't want people going down the wrong path. The auto.offset.reset config is on the consumer side, not the topic or broker side. You can add it as a custom property all day long in the Kafka configuration page but that won't help anything. This setting is driven from whatever code is consuming your topic. Also, some of the language has changed: "latest, earliest, or none" work for the new consumer API and "smallest, largest, or {any other value}" work for the old consumer API.