Web Scraper in Go, similar to BeautifulSoup
Go
Updated May 17, 2019
#
webscraper
Repositories 342
a class that uses scraped proxies to make an http GET/POST request (Python requests)
python
requests-module
requests
proxy
proxy-server
proxy-list
webscraping
webscraper
webscraper-api
recursion-problem
recursion
http-getter
http-get
http
http-proxy
http-proxy-middleware
python3
python-requests
proxy-requests
Python
Updated May 14, 2019
A command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, …
xquery
xml
html
json
xpath
cli
command-line
http
web
rest
css-selector
wget
curl
httpie
xmlstarlet
webscraper
webscraping
scraper
datascraping
headless
Pascal
Updated May 25, 2019
An R web crawler and scraper
R
Updated Mar 26, 2019
Scrapes g4g and creates PDF
Python
Updated Dec 16, 2018
A php crawler that finds emails on the internets
PHP
Updated Jan 24, 2019
A command line tool for checking your website for GDPR compliance.
JavaScript
Updated Feb 11, 2019
Cryptocurrency Historical Market Data R Package
coinmarketcap
cryptocurrencies
kaggle
crypto
dataset
cryptocurrency
webscraper
rpackage
financial
open-data
market-data
cran
historical-cryptocurrency-prices
R
Updated May 27, 2019
Speedy, lightweight web scrapper for Shutterstock.
Python
Updated May 31, 2019
This Python script will download chapters from novels availaible on wuxiaworld.com saves then into the .epub format
Python
Updated Jun 13, 2019
Python class to scrape data from rightmove.co.uk and return listings in a pandas DataFrame object
rightmove
webscraper
pandas
pandas-dataframe
csv
python
python3
data-science
data-analysis
data-mining
Jupyter Notebook
Updated Apr 2, 2019
The big downloader of concurrent downloading
C++
Updated Dec 2, 2018
Nunux Keeper core API
JavaScript
Updated Jul 4, 2019
DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based…
dotnetcore
crawler
crawling
scraping
scrapy
scrapy-crawler
entity-framework-core
ddd-architecture
csharp
webcrawler
webcrawling
webscraping
webscraper
webcrawler-htmlagilitypack
htmlagilitypack
C#
Updated Apr 4, 2019
A web scraping tool to allow freeCodeCamp campers to download their solutions.
HTML
Updated Nov 12, 2017
Good first issues
Python
Updated Jun 7, 2019
Web data extraction tool implemented as chrome extension
Python script/class to download an entire Imgur album in one go into a folder of your choice.
An automated, programming-free web scraper for interactive sites
HTML
Updated Apr 18, 2019
Yet another Python web scraping application
Python
Updated Jul 5, 2019
Library packs for google cloud functions
google
cloud
functions
gcf
pandas
numpy
selenium
chrome
serverless
gcp
webscraper
tensorflow
gcf-packs
serverless-architectures
chromium
Python
Updated Jun 11, 2019
Historical data parser for cryptos
historical-cryptocurrency-prices
daily-historical-data
webscraper
coinmarketcap
cryptocurrencies
vector
Python
Updated Jun 19, 2019
[partially working] Scrape and add anime episode stream URLs to uGet (Linux) or IDM (Windows) ~ Python3
anime
webscraper
webscraping
otaku
downloader
anime-downloader
anime-scraper
python3
scraping
streaming
stream
stream-url
idm
uget
chrome-devtools
chrome
fun
Python
Updated Mar 8, 2019
Instagram client for android for viewing posts (pic & vid) in full-screen with swipe capability without having an ins…
react
nodejs
express
recommendation-system
tensorflow
mongodb
redis
webscraper
materialize-css
rpc-api
Python
Updated Aug 21, 2018
Financial Web Scraper & Sentiment Classifier
Python
Updated Jul 18, 2018
Python command line tool to scrape polling data from RealClearPolitics and output as .csv
Python
Updated Mar 18, 2018
Python
Updated Jun 15, 2019
TagIE : Web Process automation in AutoHotKey
Updated Dec 17, 2018
A serverless, scalable service to get website description and preview deployed on AWS Lambda.
JavaScript
Updated Dec 9, 2017