An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

A Python module to bypass Cloudflare's anti-bot page.

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

Scrape data from websites using Open Graph metadata, regular HTML metadata, and a series of fallbacks.

Scrape any website, article or RSS/Atom Feed with ease!

A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library

A instagram scraper wrote in python. Similar to instagram-php-scraper.Usages are in example.py. Enjoy it!

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.

Scrape Instagram's API with Puppeteer

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges

[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library

Golang pkg to quickly return a preview of a webpage (title/description/images)

MetroLyrics API for Python

Python library for retrieving free proxies (HTTP, HTTPS, SOCKS4, SOCKS5).

scrapers for building your own image databases

A sports data scraping and analysis tool

Scrape Instagram

DuckDuckGo Image Search Resuts - Programatically download results

Python Script to Scrape Pastebin with Regex.

Extract all CSS from a given url, both server side and client side rendered.

Spider项目将会不断更新本人学习使用过的爬虫方法！！！

stitch & scrape tiles from slippy map services

This is a web crawler for pubg.op.gg, written by Ruichong Liu. 绝地求生游戏数据抓取

:package: R package to easily web scrape Glassdoor company reviews. Write up of demo:

Retrieve years of imgur.com's data without any authentication.

A simple, easy to use, scalable scraping framework written in PHP

An example project built using django 1.7 and scrapy 1.0.3