A next-generation crawling and spidering framework.
-
Updated
Nov 11, 2022 - Go
#
gocrawler
Here are 3 public repositories matching this topic...
分布式爬虫项目，本项目支持个性化定制页面解析器二次开发，项目整体采用微服务架构，通过消息队列实现消息的异步发送，使用到的框架包括：redigo, gorm, goquery, easyjson, viper, amqp, zap, go-micro，并通过Docker实现容器化部署，中间爬虫节点支持水平拓展。
-
Updated
Feb 15, 2021 - Go
Improve this page
Add a description, image, and links to the gocrawler topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the gocrawler topic, visit your repo's landing page and select "manage topics."