MCP server and CLI that crawls websites for AI agents: finds the answer page 2x more often than Crawl4AI, 10x cheaper per answer than letting Claude crawl.
vinodborole/okf-kitTurn any website into a portable, agent-ready Open Knowledge Format (OKF) bundle — no LLM required. Crawl, sync, chat offline, and serve docs to coding agents over MCP.
lovecatisgood-sudo/Free-Opensource-SEO-Screaming-Toad-not-Frog-tool-with-100million-url-crawl-potentialDJAI Academy community-built, free open-source technical SEO crawler and Screaming Frog alternative with JavaScript audits, reports, a local dashboard, and an MCP server for AI agents. Theoretical 100