In this post, I would talk about how I find the best packages and resources about Scrapy using Google BigQuery, and I wish it can help you or inspire you to find gold in your area.
This is the #10 post of my Scrapy Tutorial Series, in this Scrapy tutorial, I will show you how to write a real spider so that we can reuse the commands, tools we talked about in previous posts and the new spider would iterate the container, iterate over the pages so in the end, we could get all the quotes from Quotes to Scrape.
This is the #9 post of my Scrapy Tutorial Series, in this Scrapy tutorial, I will talk about how to define Scrapy item, how to use Scrapy item, and how to create a custom Item Pipeline to save the data of Item into DB.
Some readers might have questions when reading this scrapy tutorial series, what is the
extract method, how to use it, what if I want to iterate a note list and extract the sub-nodes. In this post, I would talk about Scrapy Selector and how to use it with iteration.