In this project, we will build a web scraper to extract data from software job postings from a public forum so that
we can count and see which technologies are the most in-demand. We will then visualize the data on a graph. This tutorial
is suitable for Python beginners.
💻 Code: https://github.com/pixegami/simple-web-scraper
👉 BeautifulSoup: https://www.crummy.com/software/BeautifulSoup/bs4/doc/
👉 MatPlotLib: https://matplotlib.org/
👉 HackerNews: https://news.ycombinator.com/
👉 Ethics in Web Scraping: https://towardsdatascience.com/ethics-in-web-scraping-b96b18136f01
📽 Video Content
00:00 Project overview
01:36 Make an HTTP request
05:49 Parse the HTTP response with "beautifulsoup"
09:06 Extract individual comments
16:04 Clean up the response text
17:16 Process the scraped content for useful data
25:05 Visualizing the data with "matplotlib"
27:43 Wrapping up