some classes from streaming provided by dependencies are overriding with different implementation    the solution is to move all classes from package org.apache.kafka to shade   my spark version is 2.4.1   here the plugin for maven build    <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>2.4.3</version> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <relocations> <relocation> <pattern>org.apache.kafka</pattern> <shadedPattern>shade.org.apache.kafka</shadedPattern> </relocation> </relocations> </configuration> </execution> </executions> </plugin>   ... View more