A collection of awesome web crawler,spider in different languages

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

A list of practical knowledge-building projects.

Web Scraper in Go, similar to BeautifulSoup

Generate and download e-books from online sources.

Faster requests on Python 3

🚀 Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy

Bulk download your favourite anime episodes from your favourite anime websites

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

OnlyFans content downloader

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

A list of scrapers from around the web.

A framework for creating semi-automatic web content extractors

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.

A universal web-util for PHP.

NBA Stats API via Basketball Reference

Fetch user's data across social media

A Reddit bot that summarizes news articles written in Spanish or English. It uses a custom built algorithm to rank words and sentences.

A simple browser/client-side web scraper.

A collection of awesome web scaper, crawler.