Scrapinghub

Repositories

• shub

  Scrapinghub Command Line Client
  Python 59 99 33 (6 issues need help) 9 Updated Jan 10, 2021

• article-extraction-benchmark

  Article extraction benchmark: dataset and evaluation scripts
  Python MIT 8 47 1 0 Updated Jan 8, 2021

• shublang

  Pluggable DSL that uses pipes to perform a series of linear transformations to extract data
  Python BSD-3-Clause 4 7 23 (1 issue needs help) 5 Updated Jan 7, 2021

• webstruct-demo

  HTTP demo for https://github.com/scrapinghub/webstruct
  Python MIT 3 4 0 3 Updated Jan 8, 2021

• jira

  Forked from pycontribs/jira

  Python JIRA Library is the easiest way to automate JIRA. Support for py27 was dropped on 2019-10-14, do not raise bugs related to it.
  Python BSD-2-Clause 627 0 0 0 Updated Jan 7, 2021

• spidermon

  Scrapy Extension for monitoring spiders execution.

  testing monitoring scraping crawling spiders hacktoberfest monitoring-tool
  Python BSD-3-Clause 62 299 31 (1 issue needs help) 12 Updated Jan 4, 2021

• scrapy-autoextract

  Scrapinghub AutoExtract API integration for Scrapy
  Python BSD-3-Clause 8 29 1 0 Updated Dec 30, 2020

• autoextract-poet

  web-poet definitions for AutoExtract
  Python 1 2 1 1 Updated Dec 30, 2020

• scrapy-poet

  Page Object pattern for Scrapy
  Python BSD-3-Clause 10 35 5 2 Updated Dec 29, 2020

• scrapinghub-autoextract

  Python clients for Scrapinghub AutoExtract API
  Python BSD-3-Clause 4 23 7 1 Updated Dec 29, 2020

• scrapy-autounit

  Automatic unit test generation for Scrapy.
  Python BSD-3-Clause 13 35 9 0 Updated Dec 28, 2020

• number-parser

  Parse numbers written in natural language
  Python BSD-3-Clause 4 18 9 1 Updated Dec 28, 2020

• extruct

  Extract embedded metadata from HTML markup

  microformats semantic-web rdfa json-ld microdata opengraph
  Python 83 557 17 (1 issue needs help) 12 Updated Dec 28, 2020

• scrapinghub-entrypoint-scrapy

  Scrapy entrypoint for Scrapinghub job runner
  Python 11 18 0 0 Updated Dec 23, 2020

• docker-images
  Dockerfile 9 29 0 3 Updated Dec 17, 2020

• dateparser

  python parser for human readable dates
  Python BSD-3-Clause 341 1,659 152 (4 issues need help) 52 Updated Dec 16, 2020

• epmdless_dist
  Erlang 0 2 0 0 Updated Dec 3, 2020

• collection-scanner

  HubStorage collection scanner library
  Python BSD-3-Clause 1 5 0 0 Updated Nov 26, 2020

• kafka-consumer-group-exporter

  Forked from braedon/prometheus-kafka-consumer-group-exporter

  Prometheus Kafka Consumer Group Exporter
  Python MIT 36 0 0 0 Updated Nov 26, 2020

• price-parser

  Extract price amount and currency symbol from a raw text string
  Python BSD-3-Clause 30 166 11 (4 issues need help) 8 Updated Nov 25, 2020

• python-crfsuite

  A python binding for crfsuite

  data-science crf crfsuite
  Python MIT 196 673 43 5 Updated Nov 19, 2020

• autoextract-examples
  Jupyter Notebook 1 2 0 1 Updated Nov 18, 2020

• splash

  Lightweight, scriptable browser as a service with an HTTP API
  Python 436 3,222 309 24 Updated Nov 18, 2020

• kafka-manager

  Forked from yahoo/CMAK

  A tool for managing Apache Kafka.

  kafka kafka-manager
  Scala Apache-2.0 2,238 3 0 0 Updated Nov 17, 2020

• crawlera-clients

  Crawlera HTTPS clients collection
  Java 0 2 0 0 Updated Nov 17, 2020

• kafka-docker

  Forked from wurstmeister/kafka-docker
  Shell Apache-2.0 2,231 0 0 0 Updated Nov 10, 2020

• autoextract-spiders

  Pre-built Scrapy spiders for AutoExtract
  Python BSD-3-Clause 8 16 2 3 Updated Nov 9, 2020

• spidyquotes

  Example site for web scraping tutorials

  playground scraping crawling tutorials web-scraping web-crawling web-scraping-tutorials
  Julia 13 23 4 2 Updated Oct 26, 2020

• scrapinghub-stack-scrapy

  Software stack with latest Scrapy and updated deps
  Dockerfile BSD-3-Clause 10 39 0 0 Updated Oct 20, 2020

• asgi_rabbitmq

  Forked from lostcitizen/asgi_rabbitmq

  RabbitMQ backend for ASGI
  Python 0 0 0 2 Updated Oct 2, 2020