Support Questions

Find answers, ask questions, and share your expertise

Submitted Storm topology fails due to class serialization error

avatar
Explorer

I am running HDP 2.2 cluster. Trying to submit Storm topology which has been tested to run in local development environment. Topology gets deployed to the cluster but fails to initialize due to some class serializaiton issue.

I am getting the following error:

2016-01-09 00:18:51 b.s.d.worker [ERROR] Error on initialization of server mk-worker
java.lang.RuntimeException: java.io.InvalidClassException: backtype.storm.task.ShellBolt; local class incompatible: stream classdesc serialVersionUID = -141067271633232418, local class serialVersionUID = 1217679306363381476
        at backtype.storm.serialization.DefaultSerializationDelegate.deserialize(DefaultSerializationDelegate.java:56) ~[storm-core-0.9.3.2.2.9.0-3393.jar:0.9.3.2.2.9.0-3393]
...

To avoid library incompatibility, I have localy copied the

storm-core-0.9.3.2.2.9.0-3393.jar

from the cluster running Storm instance. The library is used to build the topology and to submit it. Still no luck.

Anybody experienced similar issue? How do you submit your topologies to Storm?

1 ACCEPTED SOLUTION

avatar
Explorer

Issue resolved.

Solution was to direct topology sumbitting tool to the custom built Storm library as described in streamparse documentation

View solution in original post

2 REPLIES 2

avatar

Serialization errors is generally an indication of initializing any variables in spout or bolt constructors. Users should avoid this and use prepare method to initialize the variables or do any further actions on the variables.

avatar
Explorer

Issue resolved.

Solution was to direct topology sumbitting tool to the custom built Storm library as described in streamparse documentation