Pinned

1. scrapydweb Public

   Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO 👉

   Python 2.5k 472

2. logparser Public

   A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

   Python 68 18

3. scrapyd-cluster-on-heroku Public

   Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉

   Python 114 86

4. scrapyd Public

   Forked from scrapy/scrapyd

   [PR #326] Native support for basic auth 🔒 `pip install -U git+https://github.com/scrapy/scrapyd.git`, then add `username = yourusername` and `password = yourpassword` in the scrapyd.conf file. DOCS 👉

   Python 9 4

5. files Public

   Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects

   400 66

6. notes Public

   Keep on reading

   Python 4 2