Learning Apache Kafka for Beginner

[#1]

Hello All,

I have recently started my journey in Big data technologies and my background has been mainly core Java. I have a project where I need to be able to process streams of call records and correlate them for business analytics, generate dashboard and events based on specific patterns/criteria.

As part of that project I am planning to explore Kafka, can you recommend good tutorials, books explaining how Kafka works, how to deploy and code to use Kafka capabilities and optimizing it for production.

[#2]

A good start point is Hortonworks Kafka page. You'll find tutorials, explanation how Kafka works and always helpful blog with interesting articles. Enjoy!

http://hortonworks.com/hadoop/kafka/

[#3]

@dan goldberg

This is very basic but it can help to understand the kafka basic model

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_installing_manually_book/content/validate...

Basic training http://www.slideshare.net/miguno/apache-kafka-08-basic-training-verisign

[#4]

Apache Kafka is a high-throughput distributed messaging system developed by LinkedIn. Kafka is a distributed, partitioned commit log service, that provides the functionality of a messaging system with a unique design. It is written in Scala and does not follow JMS (Java Message Service) standards.

The best way to learn about Kafka is read the original design page http://kafka.apache.org/ .That will give you an overview of the motivation behind the design choices and what makes Kafka efficient. It is also a very engaging read if you are interested in systems.

In terms of adoption, Kafka is currently used in production at LinkedIn, Twitter, Tumblr, Square and a number of different companies. You can read about the uses cases that those companies found for Kafka

here : https://cwiki.apache.org/confluence/display/KAFKA/Powered+By

[#5]

@Sreenivas Adiki

I am not able to compile a program myconsumer.java which uses the following classes from kafkaimport kafka.consumer.Consumer; import kafka.consumer.ConsumerConfig; import kafka.consumer.ConsumerIterator; import kafka.consumer.KafkaStream; import kafka.javaapi.consumer.ConsumerConnector; import org.apache.spark.sql.DataFrame; import org.apache.spark.sql.SQLContext;It gives errors " cannot load or not found "is there way to resolve this . I have already tried including classpath or using javac -classpath <> myconsumer.java with no successREgardsSReeni

[#6]

I have created a complete Kafka course on Udemy (over 9000 students, 1800 reviews, 4.6 stars rating) to get started:

https://www.udemy.com/apache-kafka/?couponCode=KAFKAHDP10

It will get you started very quickly and allow you learn about the most important concepts in less than two hours. In total there are 4 hours of content!

And there are more tutorials at https://kafka-tutorials.com

Happy learning!

[#7]

I have a small article to start with kafka. I have tried to keep it simple and precise.

https://www.linkedin.com/pulse/introduction-kafka-using-nodejs-pankaj-panigrahi