admin ui for scrapy/open source scrapinghub

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO 👉

Possibly the best practice of Scrapy 🕷 and renting a house 🏡

Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉

Django based application that allows creating, deploying and running Scrapy spiders in a distributed manner

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

A django admin site for scrapy

admin ui for scrapy/open source scrapinghub

Simple Web UI for Scrapy spider management via Scrapyd

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: http://ftp.zew.de/pub/zew-docs/dp/dp18033.pdf

爬虫管理平台

Library designed to replace the SQLite backend by a MongoDB backend on Scrapy queue management

Wrapper for running Scrapyd in Heroku or locally as a service

SpiderMan Based on Scrapy, scrapyd, scrapy-API, tornado spider distributed management framework.

Various Dockerfiles.

知乎用户爬虫，使用scrapy_redis，scrapyd，gerapy等

后端基于Python的Flask和Scrapy,前端基于React,redux，采用docker部署的资讯收集站

How to set up Scrapyd cluster on Heroku

start a new (random) life | World Bank data scraper on django rest app

CLI and client library for scrapyd. Done right.

Scrapyd Dashboard

docker platform sample for scrapyd

A Python wrapper for working with Scrapyd's API.

How to set up Scrapyd cluster on Heroku

异步scrapyd api实现

Scrapy project template

my blog website