istresearch/scrapy-cluster

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

⭐ 1.2k ⑂ 320 Python MIT · 2023-11-07推送

http://scrapy-cluster.readthedocs.io/

distributed kafka python redis scraping scrapy

1.2k

Watchers

0

贡献者

0

Commits

0

Releases

17

Open Issues

2023-11-07

最近推送

原文中文

暂无 README