In this video, we'll build an end to end realtime voting system with the use of modern bidata technologies like Apache Kafka, Apache Spark and Streamlit.
Timestamps:
0:00 Introduction
1:06 The Election Flow
2:00 The System Architecture
3:02 Setting up the Project
5:16 The docker compose for the system architecture
10:10 Candidate, Party and Voters Accredditation
45:05 Voting System
1:02:03 Streaming votes from/to Kafka
1:10:43 Reading Voting Data Streams from Kafka with Pyspark
1:13:39 Spark Data preprocessing
1:15:54 Spark Aggregations
1:18:58 Writing aggregated data back to kafka
1:28:43 Realtime Votes Dashboard with Streamlit
2:14:19 Results and Live commentaries
2:16:16 Winner of the Election
2:17 31 Outro
👍 Support Our Channel:
If you find our content valuable, consider supporting us by joining our channel membership, where you'll get exclusive access to behind-the-scenes content, Q&A sessions, and much more!
https://www.youtube.com/channel/UCAEOtPgh29aXEt31O17Wfjg/join
Resources and Links
Full code: https://github.com/airscholar/realtime-voting-data-engineering
Medium Article: https://medium.com/@yusuf.ganiyu/building-a-real-time-election-voting-system-an-end-to-end-data-engineering-project-db372d0f5a39
Streamlit Documentation: https://docs.streamlit.io/
More Videos Like this: datamasterylab.com
💬 Join the Conversation:
We love hearing from you! Share your thoughts, questions, or experiences related to data engineering or this project in the comments below. Don't forget to like, subscribe, and hit the bell icon to stay updated with our latest content.
🙏 Thank You for Watching!
Remember to subscribe and hit the bell icon for notifications. Stay curious and keep exploring the fascinating world of data engineering!
#RealTimeVotingSystem #DataEngineering #ElectionTechnology #BigData #DataScience #SoftwareDevelopment #TechTutorial #Educational #ProjectShowcase #DataEngineeringProject #Pyspark #Streamlit #Kafka