Web Scraping Databases with Mechanical Soup and SQlite

Hi Everyone! In this step by step tutorial, we will extract a huge table of data from the internet and store it inside an SQLite database! To keep things simple I've chosen a Wikipedia table, but I highly encourage you to apply the same principles on data that updates a bit more frequently (for example weather forecasts) 😃 If you're curious about my IDE - I'm using Wayscript which is now available for the wide public! you no longer need an invitation, you can simply sign up with the following link: https://app.wayscript.com ⭐clone complete tutorial code⭐ https://app.wayscript.com/lairs/517c9eb3-a662-41ec-9fe8-c09b2a7559bc/public ⏰ TIMESTAMPS ⏰ *************************************** 00:00 - intro 00:34 - imports and installs 01:42 - web scraping with mechanical soup 02:20 - select HTML table elements 03:47 - extract element attributes 06:11 - find the index value of a list item 07:13 - extract multiple columns of table data 09:44 - organize extracted columns 12:44 - enumerate function 14:02 - dictionary to data frame 14:53 - create SQLite database 15:36 - create SQLite table 16:35 - insert Pandas data frame into SQlite table 17:26 - save data permanently inside database file 18:49 - thanks for watching! 💻 CODE AND IMPORTANT LINKS 💻 *************************************** ⭐ URL used in the tutorial: https://en.wikipedia.org/wiki/Comparison_of_Linux_distributions ⭐ complete code repository on Github: https://github.com/MariyaSha/WebscrapingDatabases ⭐install SQLite on Linux: sudo apt install sqlite3 ⭐install SQLite on Windows: Download the Precompiled Binaries for Windows zip file from SQLite docs: https://www.sqlite.org/download.html ⭐install SQLite on MAC or Anaconda: no need to install - you already have it! 😁 ⭐ code used in the tutorial: column_names = ["Founder", "Maintainer", "Initial_Release_Year", "Current_Stable_Version", "Security_Updates", "Release_Date", "System_Distribution_Commitment", "Forked_From", "Target_Audience", "Cost", "Status"] 📽️ RELATED TUTORIALS📽️ *************************************** 🌞 Much Better HTML table Web Scraping with Pandas: https://youtu.be/oF-EMiPZQGA 🌞 SQLite Databases for Beginners: https://youtu.be/Ohj-CqALrwk 🌞 Web Scraping Images with Mechanical Soup: https://youtu.be/drDdb1MBBfI 🌞 Web Scraping Text with Beautiful Soup: https://youtu.be/ySNSY7iiBDY