Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Learning Apache Kafka for Beginner

avatar
New Contributor

Hello All,

I have recently started my journey in Big data technologies and my background has been mainly core Java. I have a project where I need to be able to process streams of call records and correlate them for business analytics, generate dashboard and events based on specific patterns/criteria.

As part of that project I am planning to explore Kafka, can you recommend good tutorials, books explaining how Kafka works, how to deploy and code to use Kafka capabilities and optimizing it for production.

6 REPLIES 6

avatar
Expert Contributor

A good start point is Hortonworks Kafka page. You'll find tutorials, explanation how Kafka works and always helpful blog with interesting articles. Enjoy!

http://hortonworks.com/hadoop/kafka/

avatar
Master Mentor

avatar
Contributor

Apache Kafka is a high-throughput distributed messaging system developed by LinkedIn. Kafka is a distributed, partitioned commit log service, that provides the functionality of a messaging system with a unique design. It is written in Scala and does not follow JMS (Java Message Service) standards.

The best way to learn about Kafka is read the original design page http://kafka.apache.org/ .That will give you an overview of the motivation behind the design choices and what makes Kafka efficient. It is also a very engaging read if you are interested in systems.

In terms of adoption, Kafka is currently used in production at LinkedIn, Twitter, Tumblr, Square and a number of different companies. You can read about the uses cases that those companies found for Kafka

here : https://cwiki.apache.org/confluence/display/KAFKA/Powered+By

avatar
Contributor

@Sreenivas Adiki

I am not able to compile a program myconsumer.java which uses the following classes from kafkaimport kafka.consumer.Consumer; import kafka.consumer.ConsumerConfig; import kafka.consumer.ConsumerIterator; import kafka.consumer.KafkaStream; import kafka.javaapi.consumer.ConsumerConnector; import org.apache.spark.sql.DataFrame; import org.apache.spark.sql.SQLContext;It gives errors " cannot load or not found "is there way to resolve this . I have already tried including classpath or using javac -classpath <> myconsumer.java with no successREgardsSReeni

avatar
New Contributor

I have created a complete Kafka course on Udemy (over 9000 students, 1800 reviews, 4.6 stars rating) to get started:

https://www.udemy.com/apache-kafka/?couponCode=KAFKAHDP10

27438-screen-shot-2017-08-12-at-105304-pm.png

It will get you started very quickly and allow you learn about the most important concepts in less than two hours. In total there are 4 hours of content!

And there are more tutorials at https://kafka-tutorials.com

Happy learning!

avatar
New Contributor

I have a small article to start with kafka. I have tried to keep it simple and precise.

https://www.linkedin.com/pulse/introduction-kafka-using-nodejs-pankaj-panigrahi