Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Issue when deploying a new parser topology in Storm with KAFKA (Offset lags for kafka not supported for older versions)

Solved Go to solution

Issue when deploying a new parser topology in Storm with KAFKA (Offset lags for kafka not supported for older versions)

New Contributor

Hi,

I followed the steps to deploy a new data source in Metron but I am blocked to the step : deploy the new parser topology because I noticed an error in STORM Topology complaining about KAFKA version issue

After I submitted the squid parder topogoly to HOST $HOST_WITH_ENRICHMENT_TAG as root.

I went o to the Storm UI and I noticed that the new "squid" topology was an error :

Topology spouts lag error

Id Type Message
kafkaSpout KAFKA Offset lags for kafka not supported for older versions. Please update kafka spout to latest version.

9489-16-nov-16-9-47-53-am.png

Did you see this error before ?

What do I need to do to solve it ?

Thanks for your help

Maxime

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Issue when deploying a new parser topology in Storm with KAFKA (Offset lags for kafka not supported for older versions)

Guru

Metron 0.2.1BETA is based on HDP2.4, and storm 0.10.x

For HDP 2.5 you want to be using Metron 0.3.0, which has just been released. This will be built against the storm version your specified.

6 REPLIES 6

Re: Issue when deploying a new parser topology in Storm with KAFKA (Offset lags for kafka not supported for older versions)

Cloudera Employee

HDP 2.5 is actually based on Apache Storm 1.1.0-SNAPSHOT which means that some features are not yet available to official Apache Storm releases. "Topology Spout Lag" is one of them.

You can just ignore that error since it doesn't affect topology, or change the version of 'storm-kafka' to 1.0.1.2.5.0.0-1245 (NOTE: Hortonworks version) to make it work.

Hope this helps.

Re: Issue when deploying a new parser topology in Storm with KAFKA (Offset lags for kafka not supported for older versions)

New Contributor

Here some info about my SW versions :

METRON_VERSION=0.2.1BETA

Storm version : 1.0.1.2.5.0.0-1245

Re: Issue when deploying a new parser topology in Storm with KAFKA (Offset lags for kafka not supported for older versions)

Guru

Metron 0.2.1BETA is based on HDP2.4, and storm 0.10.x

For HDP 2.5 you want to be using Metron 0.3.0, which has just been released. This will be built against the storm version your specified.

Re: Issue when deploying a new parser topology in Storm with KAFKA (Offset lags for kafka not supported for older versions)

New Contributor

Hi simon, I upgraded Meton to 0.3.0 2 weeks ago and since my issue is gone.

thank you

Maxime

Re: Issue when deploying a new parser topology in Storm with KAFKA (Offset lags for kafka not supported for older versions)

New Contributor

Hi !

We are running Storm 1.0.1.2.5.0.0-1245 and Hadoop 2.5.0.0-1245, we get the same issue, any hint ?

Thanks for your help

Regards,

Vince

Re: Issue when deploying a new parser topology in Storm with KAFKA (Offset lags for kafka not supported for older versions)

Contributor

Hi Vincent.

What is the version of storm-kafka that you are using to build your topology? Can you try to upgrade it to the same version as mentioned by @Jungtaek Lim above?

Don't have an account?
Coming from Hortonworks? Activate your account here