Issue when deploying a new parser topology in Storm with KAFKA (Offset lags for kafka not supported for older versions)

[#1]

Hi,

I followed the steps to deploy a new data source in Metron but I am blocked to the step : deploy the new parser topology because I noticed an error in STORM Topology complaining about KAFKA version issue

After I submitted the squid parder topogoly to HOST $HOST_WITH_ENRICHMENT_TAG as root.

I went o to the Storm UI and I noticed that the new "squid" topology was an error :

Topology spouts lag error

Id	Type	Message
kafkaSpout	KAFKA	Offset lags for kafka not supported for older versions. Please update kafka spout to latest version.

Did you see this error before ?

What do I need to do to solve it ?

Thanks for your help

Maxime

[#2]

HDP 2.5 is actually based on Apache Storm 1.1.0-SNAPSHOT which means that some features are not yet available to official Apache Storm releases. "Topology Spout Lag" is one of them.

You can just ignore that error since it doesn't affect topology, or change the version of 'storm-kafka' to 1.0.1.2.5.0.0-1245 (NOTE: Hortonworks version) to make it work.

Hope this helps.

[#3]

Here some info about my SW versions :

METRON_VERSION=0.2.1BETA

Storm version : 1.0.1.2.5.0.0-1245

[#4 · Accepted solution]

Metron 0.2.1BETA is based on HDP2.4, and storm 0.10.x

For HDP 2.5 you want to be using Metron 0.3.0, which has just been released. This will be built against the storm version your specified.

[#5]

Hi simon, I upgraded Meton to 0.3.0 2 weeks ago and since my issue is gone.

thank you

Maxime

[#6]

Hi !

We are running Storm 1.0.1.2.5.0.0-1245 and Hadoop 2.5.0.0-1245, we get the same issue, any hint ?

Thanks for your help

Regards,

Vince

[#7]

Hi Vincent.

What is the version of storm-kafka that you are using to build your topology? Can you try to upgrade it to the same version as mentioned by @Jungtaek Lim above?