An excellent Spark package for consuming Kafka, ported from the Storm spout for Kafka.
works with all Spark and Kafka (0.8, 0.9, and 0.10) Works with Zookeeper.
Receiver based approach in this consumer is not recommended for Kafka sources.
Kafka sources have moved onto Direct Receiver model and that the model for structured streaming.
This Receiver based is much scalable and perform better than Direct stream. The Receiver mode available in Spark out of the box has some serious issue . But this consumer from Spark Packages are doing much better than Direct mode and highly adopted across the community. You can read the ReadMe file to know more details about it and how it differs from Direct Stream. https://github.com/dibbhatt/kafka-spark-consumer/blob/master/README.md