A little like that j-thing, only in Go.
Go 11.5k 846
Polite, slim and concurrent web crawler.
Go 2k 194
A simple and flexible web crawler that follows the robots.txt policies and crawl delays.
Go 774 100
tiny Go library to normalize URLs
Go 422 59
Package rehttp implements a Go HTTP transport that handles retries.
Go 192 14
Proper URL escaping as per RFC3986
Go 23 14
Loading…
