Scrapy project

itemloaders Public
Library to populate items using XPath and CSS with a convenient API

Python 32 BSD-3-Clause 13 17 5 Updated Aug 27, 2022
scrapy Public
Scrapy, a fast high-level web crawling & scraping framework for Python.

Python 44,363 BSD-3-Clause 9,704 515 (30 issues need help) 279 Updated Aug 26, 2022
scrapy.org Public
The scrapy.org website

HTML 48 154 0 1 Updated Aug 26, 2022
parsel Public
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

Python 805 BSD-3-Clause 129 30 17 Updated Aug 26, 2022
w3lib Public
Python library of web-related functions

Python 354 BSD-3-Clause 98 16 (1 issue needs help) 6 Updated Aug 11, 2022
itemadapter Public
Common interface for data container classes

Python 41 BSD-3-Clause 8 4 2 Updated Aug 2, 2022
scrapyd Public
A service daemon to run Scrapy spiders

Python 2,510 BSD-3-Clause 546 30 19 Updated Jul 12, 2022
protego Public
A pure-Python robots.txt parser with support for modern conventions.

DIGITAL Command Language 31 BSD-3-Clause 21 2 (1 issue needs help) 1 Updated Jul 7, 2022
xtractmime Public
https://mimesniff.spec.whatwg.org/ implementation for Python

Python 10 BSD-3-Clause 3 0 0 Updated Jul 3, 2022
quotesbot Public
This is a sample Scrapy project for educational purposes

Python 1,152 MIT 719 0 6 Updated May 26, 2022

Pinned