Michael Yin


Python Developer - Tech Writer

Projects


Torchbox Projects


torchbox-header.jpg

In May, 2018, I am hired as a remote developer by Torchbox (Creator of Wagtail CMS)

I am responsible to build web projects based on Wagtail CMS.

The biggest feature in this project is that we need to build a multiform system for the client, the form fields on form page need to be editable (no coding required). And some form fields have some custom features

We implement this feature based on StreamField and FormBuilder of Wagtail, and we call it StreamForm. We also write some javascript to increase the user experience. Clients are very satisfied with it.

My Guest Post On Wagtal's official site. Add a Wagtail blog app to your Django project

StyleSage


stylesage-header.png

StyleSage is a big data company in fashion industry, they need to collect fashion data from dozens of e-commerce websites such as hm, Zara and etc.

To collect huge data in a quick and efficient way, we build our web scraping system based on Scrapy framework.

Because we do not rely on the headless browser so our scraping speed can be guaranteed. At the same time, we build and maintain proxy pool, backend no-sql DB to save, pre-process scraped data.

I mainly solved these problems in this project.

  1. Analyze web requests and implement in Scrapy.
  2. Analyze javascript code & JSON code to extract data.
  3. Avoid spiders being banned by sending too many requests.
  4. How to quickly review data by writing custom pipeline and some stats tool such as Pandas.

ScrapingClub


scrapingclub-header.png

ScrapingClub is my side project that helps newbie developers to learn web scraping in a better way.

Most blog posts about Web Scraping talks about how to crawl a specific website or pages, but I insist it makes more sense to help people to learn how to analyze the website and choose the right way to get the job done well.

That is why I build ScrapingClub, my goal is to try to break down a complex web scraping mission such as crawling a bunch of websites to some small tasks so people can learn how to solve them step by step.

Right now it includes 10 practical web scraping exercises already.

Wagtail Bootstrap Blog


wagtail-bootstrap-blog-header.png

Wagtail Bootstrap Blog is developed exclusively for Wagtail Tutorial Build Blog With Wagtail CMS, which shows people how to create a Wagtail blog using Bootstrap step by step. You can also import it into your Django project to quickly add professional blog function based on Wagtail.

I wish this open source project can help people to learn Wagtail CMS, which is a very powerful framework in Django ecosystem.

My Tutorial Series


Wagtail Tutorial Series

This Wagtail tutorial series teaches you how to create a standard blog using Wagtail CMS step by step.

Scrapy Tutorial Series: Web Scraping Using Python

This Scrapy Tutorial Series would teach you to build web scraper using Scrapy in Python.

My Books


Build Blog With Wagtail CMS

This book will show you how to build a standard blog using Wagtail CMS.

Ultimate Guide To Scrapy

This book will help you learn web scraping using Scrapy in Python3.

About


My name is Michael Yin, I am a full stack developer from China. Right now I am living in Wuhan, China with my wife.

I am working with Python and Javascript to create web app and web scraping program.

I am available to hire for remote jobs

Contact Me