@Paul Hargis any pointers on how to set "auto.offset.reset" in storm ? @Artem Ervits if I use "kafka.api.OffsetRequest.EarliestTime();" everytime I restart topology will it consume old data(I mean from beginning) ? My use case is this: My topology was down for over 2 weeks, but kafka has only 1 weeks data. so I am thinking that when I start the topology it will not find the offset. Should I use "kafka.api.OffsetRequest.EarliestTime();" ? update: tried restarting the topoloy I am using OpaqueTridentKafkaSpout, getting storm.kafka.UpdateOffsetException (using storm 0.9.4) checked KafkaConfig it has public boolean useStartOffsetTimeIfOffsetOutOfRange = true; public long startOffsetTime = kafka.api.OffsetRequest.EarliestTime(); So what I am not getting is that why it is not setting offset to earliest and failing with UpdateOffsetException instead 😞 Could you guys please help ? ... View more

@Sanjeev Verma You can use Config conf = new Config(); conf.put(key, val); conf.put(key1, val1); e.g redis config, etc. and then you can use this in prepare method of bolts(in case of trident in functions/ filters, etc). Hope this helps. ... View more

@Narendra Bidari for trident : It maintains offset in zookeeper folder [stream-name] So I think stream name must be acting as consumer group id. ... View more

Hey, @ggolani Thanks for the reply. TridentKafkaConfig spoutConf = new TridentKafkaConfig(zk, "test-topic",UUID.randomUUID().toString()) We have already tried this and seems not be working. The javadoc says that the 3rd param is clientId. Is this same as group id ? ... View more