Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

I am getting InvalidClassException using storm on HDP 2.5

Solved Go to solution
Highlighted

I am getting InvalidClassException using storm on HDP 2.5

Hi,

I am having HDP 2.5 and storm core version as storm-core-1.0.1.2.5.3.0-37.jar. My code was working with Local Cluster but its failing when using StormSubmitter.submitTopology. It gives me InvalidClassException. Below is complete stack.

One more observation that jar gives error while executing first time and run afterwards. I searched and implemented as below, but nothing worked. Please help me.

java.lang.RuntimeException: java.io.InvalidClassException: twitter4j.MediaEntityJSONImpl; local class incompatible: stream classdesc serialVersionUID = 1571961225214439778, local class serialVersionUID = 3609683338035442290
at org.apache.storm.serialization.SerializableSerializer.read(SerializableSerializer.java:58) ~[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]

1. added serialVersionUID with value 1571961225214439778L, 3609683338035442290, 1L but nothing worked

2. Implemented Serializable, Externalizable in my POJOs

3. Since I am using storm-core-1.0.1.2.5.3.0-37.jar & HDP 2.5 so added maven dependency in pom with scope as provided which matches it, but did not worked.

4. Added no-arg constructors.


1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: I am getting InvalidClassException using storm on HDP 2.5

Super Mentor

@parag dharmadhikari

Looks like your class "twitter4j.MediaEntityJSONImpl" version is not compatible. So please check if you can get a different version of JAR that contains the "twitter4j.MediaEntityJSONImpl" class to see if it fixed the issue.

If you still face the issue then can you please share the list of JARs that your application is using? Better if you can share the whole "pom.xml" to see of the dependency is OK.

View solution in original post

3 REPLIES 3
Highlighted

Re: I am getting InvalidClassException using storm on HDP 2.5

Super Mentor

@parag dharmadhikari

Looks like your class "twitter4j.MediaEntityJSONImpl" version is not compatible. So please check if you can get a different version of JAR that contains the "twitter4j.MediaEntityJSONImpl" class to see if it fixed the issue.

If you still face the issue then can you please share the list of JARs that your application is using? Better if you can share the whole "pom.xml" to see of the dependency is OK.

View solution in original post

Highlighted

Re: I am getting InvalidClassException using storm on HDP 2.5

thanks :) It worked. I am using twitter4j 4.0.4 now.

Re: I am getting InvalidClassException using storm on HDP 2.5

@Jay SenSharma

Thanks Jay for your response. I will try your suggestion. Mean while my POM is as below,

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>
<groupId>com.rolta</groupId>
<artifactId>stormtwitter</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.7</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.7</version>
</dependency>
<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
<version>2.9.9</version>
</dependency>
<dependency>
<groupId>org.twitter4j</groupId>
<artifactId>twitter4j-core</artifactId>
<version>4.0.6</version>
</dependency>
<dependency>
<groupId>org.twitter4j</groupId>
<artifactId>twitter4j-stream</artifactId>
<version>4.0.6</version>
</dependency>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-core</artifactId>
<version>1.0.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.opennlp</groupId>
<artifactId>opennlp-tools</artifactId>
<version>1.8.1</version>
</dependency>
<dependency>
<groupId>org.apache.solr</groupId>
<artifactId>solr-solrj</artifactId>
<version>6.6.0</version>
</dependency>
<dependency>
<groupId>com.jolira</groupId>
<artifactId>onejar-maven-plugin</artifactId>
<version>1.4.4</version>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
<excludes>
<exclude>org.slf4j:slf4j-log4j12:*</exclude>
<exclude>log4j:log4j:jar:</exclude>
<exclude>org.slf4j:slf4j-simple:jar</exclude>
<exclude>org.apache.storm:storm-core</exclude>
</excludes>
</artifactSet>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.rolta.storm.topology.Topology</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>

</plugins>
</build>

</project>

Don't have an account?
Coming from Hortonworks? Activate your account here