Archive Team

Repositories

• pastebin-grab

  Archiving pastebin
  Lua Unlicense 2 5 0 0 Updated May 12, 2021

• liveleak-grab

  Archiving liveleak.com
  Lua Unlicense 0 2 0 0 Updated May 11, 2021

• archiveteam-megawarc-factory

  Some scripts to process ArchiveTeam uploads
  Shell 6 2 1 1 Updated May 10, 2021

• grab-site

  The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

  crawler spider archiving crawl warc
  Python 78 699 58 (20 issues need help) 6 Updated May 10, 2021

• ludios_wpull

  wpull fork with fixes and faster parsing using html5-parser; used by grab-site; should go away when wpull is similarly improved
  HTML GPL-3.0 2 14 12 1 Updated May 10, 2021

• megawarc

  Forked from alard/megawarc

  Nondestructive warc-in-tar to warc conversion
  Python 5 4 1 0 Updated May 8, 2021

• bintray-grab

  Forked from OrIdow6/bintray-grab
  Lua 2 0 0 0 Updated May 8, 2021

• liveleak-items

  Managing items for liveleak-grab.
  3 2 0 0 Updated May 8, 2021

• urls-grab

  Archiving URLs (outlinks) from a variety of sources.
  Lua Unlicense 3 0 0 0 Updated May 5, 2021

• reddit-grab

  Grabbing everything from reddit.
  Lua Unlicense 5 12 0 0 Updated May 5, 2021

• yahooanswers-grab

  Saving all questions and answers from Yahoo! Answers.
  Lua Unlicense 5 47 3 0 Updated May 4, 2021

• wget-lua

  Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

  crawler spider lua crawling archiving wget crawl
  C GPL-3.0 7 40 5 1 Updated May 4, 2021

• grab-base-df

  Base Dockerfile for warrior project grab scripts
  Dockerfile MIT 2 2 0 0 Updated May 4, 2021

• bintray-items

  Managing items for bintray-grab.
  0 0 0 0 Updated May 1, 2021

• seesaw-kit

  Making a reusable toolkit for writing seesaw scripts
  Python 21 57 44 2 Updated Apr 30, 2021

• zstd-dictionary-trainer

  Training ZSTD dictionaries for use in ZST WARCs.
  Python 0 0 0 0 Updated Apr 26, 2021

• urls-sources

  Sources for urls-grab.
  0 0 1 0 Updated Apr 23, 2021

• mediafire-items

  Managing items for mediafire-grab.
  Roff 3 1 1 0 Updated Apr 22, 2021

• telegram-items

  Managing items for telegrab-grab.
  0 0 0 0 Updated Apr 22, 2021

• Ubuntu-Warrior

  Scripts to build and boot warrior virtual machine containing Docker
  Shell Unlicense 5 23 6 1 Updated Apr 22, 2021

• periscope-grab

  Archiving periscope
  Lua Unlicense 0 1 0 0 Updated Apr 21, 2021

• yahooanswers-items

  Managing items for yahoo-answers-grab.
  1 0 0 0 Updated Apr 16, 2021

• warrior-dockerfile

  A Dockerfile for the ArchiveTeam Warrior
  Shell 44 144 1 0 Updated Apr 15, 2021

• ArchiveBot

  ArchiveBot, an IRC bot for archiving websites

  javascript ruby python irc archiving haxe
  Python MIT 53 232 130 14 Updated Apr 11, 2021

• google-sites-grab

  Archiving Google Sites Classic.
  Lua Unlicense 4 3 2 0 Updated Apr 10, 2021

• mediafire-grab

  Archiving mediafire.com URLs.
  Lua Unlicense 2 2 0 0 Updated Apr 10, 2021

• github-grab

  Archiving GitHub
  Lua Unlicense 2 2 0 0 Updated Apr 10, 2021

• webs-grab

  Archiving webs.com
  Lua Unlicense 1 3 0 0 Updated Apr 10, 2021

• webs-items

  Managing items for webs-grab.
  0 0 0 0 Updated Mar 27, 2021

• super-mario-maker-bookmarks-grab

  Forked from OrIdow6/super-mario-maker-bookmarks-grab
  Lua Unlicense 1 0 0 1 Updated Mar 26, 2021