Finally I am able to resolve this issue - it is more related to improper usage of kafka dependencies in my pom.xml   This is what I did : 1. Created a new Cloudera Quickstart VM 5.8   2. Installed Kafka by following the commands mentioned in https://www.cloudera.com/documentation/kafka/latest/topics/kafka_packaging.html   In the above HTML link read carefully on maven dependency we have to use. Also check below pom.xml where kafka_clients is excluded and added from Cloudera Repository   3. In my maven pom.xml I have below configuration   Please note below kafka_clients exclusion while defining spark-streaming-kafka     <repositories> <repository> <id>cloudera</id> <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url> </repository> </repositories>   <properties> <java.version>1.8</java.version> <scala.minor.version>2.10</scala.minor.version> <scala.complete.version>${scala.minor.version}.6</scala.complete.version> <spark.version>1.6.1</spark.version> </properties> .....   <dependencies> ............... <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-streaming-kafka_${scala.minor.version}</artifactId> <version>${spark.version}</version> <exclusions> <exclusion> <groupId>org.apache.kafka</groupId> <artifactId>kafka-clients</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.kafka</groupId> <artifactId>kafka-clients</artifactId> <version>0.9.0-kafka-2.0.1</version> </dependency> ............... <dependencies>   Hope this will help others   Thanks, TrainingMyHobby   ... View more