Apache Kafka is a distributed stream processing software developed by LinkedIn and written in Scala and Java.
Chapters
0:00 Intro
2:32 Kafka Broker
5:39 Kafka Producer
7:11 Kafka Consumer
8:34 Kafka Partitions
12:50 Queue vs Pub-Sub
16:47 Consumer Group
23:06 ZooKeeper
29:45 Coding Example
1:02:48 Kafka Pros-Cons
1:17:00 Summary
Download slides here
https://payhip.com/b/Y0Cq
(Members get all slides for all my video courses for free)
In this video I want to focus on the basics of Kafka, talk about how it works give finally spin up a Kafka cluster and write a producer and a consumer.
Kafka Components
_ producer
_consumer
_topic
_partition
How Kafka works
Example ( Ride sharing )
Spin Kafka cluster
Write a producer
Write a consumer
Kafka Applications
Pros and Cons
Source Code
https://github.com/hnasr/javascript_playground/tree/master/kafka
Docker commands spin up kafka
//if you ever mess up or you want to start fresh just stop and remove the containers and rerun the command
docker stop zookeeper kafka
docker rm zookeeper kafka
docker run --name zookeeper -p 2181:2181 -d zookeeper
docker run -p 9092:9092 --name kafka -e KAFKA_ZOOKEEPER_CONNECT=husseinmac:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://husseinmac:9092 -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 -d confluentinc/cp-kafka
Get my fundamentals of database engineering course link redirects to udemy with coupon applied (https://database.husseinnasser.com)
Resources
KIP 500 (Remove Zookeeper)
https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
🏭 Backend Engineering Videos in Order
https://backend.husseinnasser.com
💾 Database Engineering Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2
🎙️Listen to the Backend Engineering Podcast
https://husseinnasser.com/podcast
Gears and tools used on the Channel (affiliates)
🖼️ Slides and Thumbnail Design
Canva
https://partner.canva.com/c/2766475/647168/10068
🎙️ Mic Gear
Shure SM7B Cardioid Dynamic Microphone
https://amzn.to/3o1NiBi
Cloudlifter
https://amzn.to/2RAeyLo
XLR cables
https://amzn.to/3tvMJRu
Focusrite Audio Interface
https://amzn.to/3f2vjGY
📷 Camera Gear
Canon M50 Mark II
https://amzn.to/3o2ed0c
Micro HDMI to HDMI
https://amzn.to/3uwCxK3
Video capture card
https://amzn.to/3f34pyD
AC Wall for constant power
https://amzn.to/3eueoxP
Stay Awesome,
Hussein