Hey, what's up everyone? Welcome back to another video! I'm super excited for this one. We're doing another complete Python Pandas tutorial walkthrough. Five years have passed since the last iteration, and both the library and my knowledge have evolved. We'll cover all the basics and advanced techniques to analyze and manipulate tabular data with Pandas. Whether you're a beginner or an experienced user looking to level up, there's something here for everyone. Let's dive in!
What We’ll Cover:
- Setting up your environment
- Introduction to DataFrames
- Loading data from CSV, Excel, Parquet, and more
- Accessing and manipulating data
- Filtering, adding, and removing columns
- Handling missing values
- Aggregating data with GroupBy and Pivot Tables
- Advanced functionalities like shift, rank, and rolling functions
- Exploring the new features in Pandas 2.0
- Using AI tools like GitHub Copilot and ChatGPT to enhance your workflow
Links Mentioned
GitHub Repo (code): https://github.com/KeithGalli/complete-pandas-tutorial
Pandas 2.0 Blog Post: https://datapythonista.me/blog/pandas-20-and-the-arrow-revolution-part-i
Datetime percentage format cheat sheet: https://strftime.org/
Kaggle NOC dataset: https://www.kaggle.com/datasets/heesoo37/120-years-of-olympic-history-athletes-and-results
Videos Mentioned!
Olympic Data Cleaning Tutorial: https://www.youtube.com/live/oad9tVEsfI0?si=QeIp8AGcGqrYgzBK
Regex Tutorial: https://youtu.be/vsa9GGzMFXQ?si=tH6vaaKJm5o4Ud93
100 Pandas Problems: https://www.youtube.com/watch?v=i7v2m-ebXB4&t=47s
Practice!
StrataScratch: https://www.stratascratch.com/?via=keith
Analyst Builder: https://www.analystbuilder.com/?via=keith
-------------------------
Video Timeline!
0:00 - Video Overview
1:11 - Getting Started with Python Pandas | Google Colab
1:21 - Getting Started with Python Pandas | Local Environment Setup (Cloning code, using virtual environment, VS Code)
3:58 - Intro to Dataframes | Creating DataFrames, Index/Columns, Basic Functionality
8:25 - Loading in DataFrames from Files (CSV, Excel, Parquet, etc.)
13:42 - Accessing Data | .head() .tail() .sample()
15:28 - Accessing Data | .loc() .iloc()
19:20 - Setting DataFrame Values w/ loc() & iloc()
20:20 - Accessing Single Values | .at() .iat()
21:11 - Accessing Data | Grab Columns, Sort Values, Ascending/Descending
23:01 - Iterating over a DataFrame (df) with a For Loop | df.iterrows()
24:12 - Filtering Data | Syntax Options, Numeric Values, Multiple Conditions
27:58 - Filtering Data | String Operations, Regular Expressions (Regex)
33:09 - Filtering Data | Query Functions
34:20 - Adding / Removing Columns | Basics, Conditional Values, Math Operations, Renaming Columns
41:40 - Adding / Removing Columns | String Operations, Datetime (pd.to_datetime) Operations
46:38 - Saving our Updated DataFrame (df.to_csv, df.to_excel, df.to_parquet, etc)
47:14 - Adding / Removing Columns | Using Lambda & Custom Functions w/ .apply()
50:42 - Merging & Concatenating Data | pd.merge(), pd.concat(), types of joins
58:33 - Handling Null Values (NaNs) | .fillna() .interpolate() .dropna() .isna() .notna()
1:04:05 - Aggregating Data | value_counts()
1:05:47 - Aggregating Data | Using Groupby - groupby() .sum() .mean() .agg()
1:08:24 - Aggregating Data | Pivot Tables
1:10:28 - Groupby combined with Datetime Operations
1:14:38 - Advanced Functionality | .shift() .rank() .cumsum() .rolling()
1:22:10 - New Functionality | Pandas 1.0 vs Pandas 2.0 - pyarrow
1:25:29 - New Functionality | GitHub Copilot & OpenAI ChatGPT
1:32:05 - What Next?? | Continuing your Python Pandas Learning…
-------------------------
Follow me on social media!
Instagram | https://www.instagram.com/keithgalli/
Twitter | https://twitter.com/keithgalli
TikTok | https://tiktok.com/@keithgalli
-------------------------
Learn data skills with hands-on exercises & tutorials at Datacamp!
https://datacamp.pxf.io/c/3588040/1012793/13294
Practice your Python Pandas data science skills with problems on StrataScratch!
https://stratascratch.com/?via=keith
Join the Python Army to get access to perks!
YouTube - https://www.youtube.com/channel/UCq6XkhO5SZ66N04IcPbqNcw/join
Patreon - https://www.patreon.com/keithgalli
*I use affiliate links on the products that I recommend. I may earn a purchase commission or a referral bonus from the usage of these links.