Code Repositories
Find and share code repositories
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Super Guru
Repo Description

An excellent Spark package for consuming Kafka, ported from the Storm spout for Kafka.

https://spark-packages.org/package/dibbhatt/kafka-spark-consumer

works with all Spark and Kafka (0.8, 0.9, and 0.10) Works with Zookeeper.

Repo Info
Github Repo URL https://github.com/dibbhatt/kafka-spark-consumer
Github account name dibbhatt
Repo name kafka-spark-consumer
1,526 Views
0 Kudos
Comments
Expert Contributor

Receiver based approach in this consumer is not recommended for Kafka sources.

Kafka sources have moved onto Direct Receiver model and that the model for structured streaming.

New Contributor

This Receiver based is much scalable and perform better than Direct stream. The Receiver mode available in Spark out of the box has some serious issue . But this consumer from Spark Packages are doing much better than Direct mode and highly adopted across the community. You can read the ReadMe file to know more details about it and how it differs from Direct Stream. https://github.com/dibbhatt/kafka-spark-consumer/blob/master/README.md

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎12-18-2016 03:30 AM
Updated by:
 
Contributors
Top Kudoed Authors