Web Scraping Exercises are now available

Last updated on by MichaelYin

Web Scraping Exercises is a collection which helps people learn Web Scraping. People can use it to improve Web Scraping skills by solving the practical Exercises.

They are available at ScrapingClub. It is a website which includes many web scraping exercise and high-quality web scraping tutorials.

Why I create this project

Most blog posts about Web Scraping talks about how to crawl a specific website or pages, but I insist it makes more sense to help people to learn how to analyze the website and choose the right way to get the job done well.

That is why I build Web Scraping exercises, my goal is to try to break down a complex web scraping mission such as crawling a bunch of websites to some small tasks so people can learn how to solve them step by step. What is more, if they have trouble solving the exercises, they can ask for help with more detail instead of "I have trouble crawling the website, please help!".

Who might need this project

Any people who want to learn web scraping, test the web scraping skills or want to make it for fun might need this project.

How it works

You will see many product detail pages and list pages on ScrapingClub. For example, Two product detail pages might look the same but use different ways to process the data. People should figure out the way and write a spider to extract data.

Short descriptions are at the top of each exercise page, which can help you understand what needs to do. And the tips or links can help you learn web scraping better. Below is a screenshot of one of the exercises.

You can use it to learn Web Scraping with any relevant tech using any language such as Python, Javascript, PHP. For example, you can build a web spider to crawl list page and product page using a PHP script to solve this exercise. Recursively Scraping pages

What if you have trouble completing the exercise?

Later I will create a project on Github hosting the solution code, and I will also write articles in more detail, please feel free to send me the message to let me know your thoughts.

Table Of Contents

I will add more in the next weeks, and you can subscribe to Mailing List to keep updated.

Take a look at the Web Scraping Exercises

Hire Me For Your Project

My hourly rate is $30 per hour, I can help you solve specific Wagtail/Django, Scrapy problem and provide 1:1 programming help.

Contact Me For a FREE Evaluation