爬虫

🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.

Crawlee — The web scraping and browser automation library that helps you build reliable crawlers. Fast.

A service daemon to run Scrapy spiders

Redis-based components for Scrapy.

Collection of China illegal cases about web crawler 本项目用来整理所有中国大陆爬虫开发者涉诉与违规相关的新闻、资料与法律法规。致力于帮助在中国大陆工作的爬虫行业从业者了解我国相关法律，避免触碰数据合规红线。 [AD]中文知识图谱门户

🔮 A Node.js scraper for humans.

开源的爬虫框架