The collections for different platforms to apply the python crawler and scrapy to extract information and also present different scraping methods

SoQues ❓: A python project to scrape questions from StackOverflow using scrapy and store them in a MongoDB Database 🗂 using pymongo.

Crawler to scrape book name , price, genre and availability from a website.

Some useless Python scripts for fun.

Data Scraping

Website crawler written in JavaScript.

Crawling a site without sitemap

A data crawler tool

Simple crawler using apache nutch and elasticsearch

python爬虫

A webscrapper that can scrape bbcnews using Scrapy and mongoDB

A crawler program to extract all of the data and the price for symbols in the global stock exchange.

拉勾网全站职位抓取CrawlSpider

scrapy框架爬取51job(scrapy.Spider)，智联招聘(扒接口)，拉勾网(CrawlSpider)

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章