#

scraping

Here are 4,164 public repositories matching this topic...

scrapy

scrapy / scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

python crawler framework scraping crawling hacktoberfest

Updated Jan 5, 2023
Python

gocolly / colly

Elegant Scraper and Crawler Framework for Golang

go golang crawler scraper framework spider scraping crawling

Updated Jan 5, 2023
Go

psf / requests-html

Pythonic HTML Parsing for Humans™

python html http scraping requests kennethreitz beautifulsoup lxml css-selectors pyquery

Updated Dec 8, 2022
Python

code4craft / webmagic

A scalable web crawler framework for Java.

java crawler framework scraping

Updated Dec 20, 2022
Java

crawlee

apify / crawlee

Crawlee—A web scraping and browser automation library for Node.js that helps you build reliable crawlers. Fast.

npm crawler scraper automation web-crawler headless scraping crawling javascript-library web-scraping web-crawling headless-chrome rpa apify puppeteer

Updated Jan 6, 2023
TypeScript

curlconverter / curlconverter

Convert cURL commands to Python, JavaScript, Java, C#, PHP, Go, Dart, R, Ruby, Rust, MATLAB, Elixir, CFML, Ansible or JSON

python java go dart rust php http tree-sitter r curl matlab scraping developer-tools python-requests

Updated Jan 6, 2023
TypeScript

tabulapdf / tabula

Tabula is a tool for liberating data tables trapped inside PDF files

pdf csv excel scraping tables

Updated May 3, 2022
CSS

lorien / awesome-web-scraping

List of libraries, tools and APIs for web scraping and data processing.

crawler spider scraping crawling web-scraping captcha-recaptcha webscraping crawling-framework scraping-framework captcha-bypass scraping-tool crawling-tool scraping-python crawling-python

Updated Dec 31, 2022
Makefile

yujiosaka / headless-chrome-crawler

Distributed crawler powered by Headless Chrome

jquery crawler chrome scraper promise scraping crawling chromium headless-chrome puppeteer

Updated Nov 11, 2022
JavaScript

ferret

MontFerret / ferret

Declarative web scraping

go cli golang crawler chrome data-mining scraper library tool dsl scraping crawling query-language scraping-websites hacktoberfest cdp hacktoberfest2021

Updated Jan 4, 2023
Go

autoscraper

alirezamika / autoscraper

Sponsor

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

python crawler machine-learning scraper automation ai scraping artificial-intelligence web-scraping scrape webscraping webautomation

Updated Jan 1, 2023
Python

sparklemotion / mechanize

Mechanize is a ruby library that makes automated web interaction easy.

ruby web scraping

Updated Jun 9, 2022
Ruby

ultrafunkamsterdam / undetected-chromedriver

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

testing chrome automation webdriver browser captcha scraping selenium navigator python3 cloudflare chromedriver anti-bot bot-detection cloudflare-bypass distil anti-detection

Updated Jan 1, 2023
Python

khuyentran1401 / Data-science

Sponsor

Collection of useful data science topics along with articles, videos, and code

python data-science machine-learning natural-language-processing time-series scraping data-visualization artificial-intelligence data-analysis articles

Updated Jan 4, 2023
Jupyter Notebook

fake-useragent / fake-useragent

Up-to-date simple useragent faker with real world database

python agent user-agent scraping fake faker python3 user useragent user-agent-spoofer useragent-scraper

Updated Dec 9, 2022
HTML

aapatre / Automatic-Udemy-Course-Enroller-GET-PAID-UDEMY-COURSES-for-FREE

Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY COURSES, ABSOLUTELY FREE!

python scraper scraping selenium python3

Updated Dec 29, 2022
Python

panther

symfony / panther

A browser testing and web crawling library for PHP and Symfony

php webdriver symfony scraping selenium chromedriver selenium-webdriver hacktoberfest e2e-testing

Updated Nov 28, 2022
PHP

NikolaiT / GoogleScraper

A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.

python search-engine crawler scraping search-engines search-engine-optimization

Updated Jul 3, 2021
HTML

emadehsan / thal

Getting started with Puppeteer and Chrome Headless for Data Mining

nodejs mongodb mongoose scraping chrome-headless puppeteer

Updated Oct 28, 2020
JavaScript

lorien / grab

Web Scraping Framework

python crawler framework spider asynchronous network python-library scraping crawling http-client python3 web-scraping pycurl urllib3

Updated Dec 28, 2022
Python

Improve this page

Add a description, image, and links to the scraping topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the scraping topic, visit your repo's landing page and select "manage topics."