Build a Live Air Quality Dashboard with Python and Plotly Dash! | Beginner Data Engineering Project

Try CodeCrafters for free using my referral link: https://app.codecrafters.io/join?via=trentpark8800

Welcome to this beginner-friendly data engineering project where we build a live air quality dashboard from scratch using Python, Plotly Dash, and DuckDB! 🌍📊

In this tutorial, I'll take you through the entire pipeline:
1. Data Extraction: Fetching real-time air quality data from OpenAQ.
2. Data Transformation: Cleaning and processing data using DuckDB for fast and efficient handling.
3. Data Visualization: Creating a sleek and interactive dashboard with Plotly Dash.

Whether you're new to data engineering or looking to sharpen your skills, this project is great for learning the basics of:
• Building a functional data pipeline 🛠️
• Working with live data feeds ⏱️
• Designing interactive dashboards 🎨

🔑 Key Topics Covered:
• Setting up your Python environment
• Using DuckDB to manage and query data
• Creating dynamic visuals with Plotly Dash
• Tips and tricks for handling real-world data challenges

🗂️ Resources:
• Code Repository: https://github.com/trentpark8800/air-quality-pipeline
• Open AQ: https://openaq.org
• DuckDB Documentation: https://duckdb.org
• Plotly Dash: https://dash.plotly.com/
• Dbeaver: https://dbeaver.io/download/
• Bbox Finder: http://bboxfinder.com/

If you found this video helpful, don’t forget to like, comment, and subscribe for more data engineering projects!

📖 Chapters:
0:00 Intro
2:10 The OpenAQ Project
5:55 Project Setup
11:50 Location Data Extraction
26:15 DuckDB and S3 Setup
43:25 Data Quality Check
53:18 Data Product Creation
01:19:00 Plotly Dash Dashboard Setup
01:49:44 End of Part 1
01:52:19 Database Manager Script
02:26:43 Data Extraction Script
03:10:00 Data Transformation Script
03:22:46 Another Look at Data Quality
03:31:30 Refining The Dashboard
04:10:28 Testing With a New Data Point

💵SUPPORT THE CHANNEL
- CodeCrafters Affiliate: https://app.codecrafters.io/join?via=trentpark8800
- Become a channel member: https://www.youtube.com/channel/UCY5ZuZAWVUBzkDXQr5b84sQ/join
- Direct, once off transfers: https://www.paypal.com/paypalme/trentdoesmath