LxL my8100

Pinned

1. scrapydweb

   Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Email notice, and Mobile UI. DEMO 👉

   Python 1.1k 195

2. logparser

   A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

   Python 22 6

3. scrapyd-cluster-on-heroku

   Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉

   Python 60 14

4. scrapyd

   Forked from scrapy/scrapyd

   [PR #326] Native support for basic auth 🔒 `pip install -U git+https://github.com/scrapy/scrapyd.git`, then add `username = yourusername` and `password = yourpassword` in the scrapyd.conf file. DOCS 👉

   Python 1 2

5. files

   Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects

   314 30

6. notes

   Keep on reading

   Python 1