Web browsing superpowers for AI agents - one MCP server to scrape, crawl, extract, map, search, batch, research & monitor the web. Self-hosted, no API keys. Free Firecrawl alternative.
View on GitHub ↗Homepage ↗🔥 PyreCrawl — Web Browsing Superpowers for Your AI Agent
One command gives any AI agent the whole web. Scrape, extract, crawl, map, and search — self-hosted, no API keys, no rate limits, no subscription.
PyreCrawl speaks MCP (Model Context Protocol), the standard tool interface for Claude, Cursor, VS Code, Codex, OpenCode, Hermes, and any MCP-compatible agent.
A smart auto-fallback ladder always picks the cheapest method that succeeds:
Tool What it does ------ scrape(url, prefer="auto") Single URL → LLM-ready markdown extract(url, schema) Scrape + structured extraction (JsonCss schema) mapsite(root, includepattern=None, limit=200) Enumerate all internal URLs crawl(root, maxpages=5, prefer="auto", includepaths=None, excludepaths=None, maxdepth=0) Multi-page crawl with path filters + true BFS depth document(url) PDF/DOCX/PPTX → markdown (no browser, optional [docs] extras) search(query, limit=10) Web search via DuckDuckGo HTML (no API key) searchpapers(query, limit=8, source="arxiv", category=None) Academic search via arXiv + Crossref (no API key) — feed pdfurl into document batchscrape(urls[], ...) Many URLs in ONE call — parallel, deduped, cache-aware deepresearch(query, limit=5, scrapetop=3) Search → evidence pack with [n] citations (no LLM synthesis — your agent does that) monitor(url, action, cssselector=None) Change detection with persisted snapshots + unified diff session(session, action, ...) Persistent browser session (cookies kept) — login walls, multi-step flows, screenshots cache(action) Inspect/clear/enable/disable the HTTP response cache health() Versions +
From the project README.
Add the radar badge to your README — it shows your project was picked up by MCP Radar and links to this page:
[](https://mcp.liqiwa.com/s/SanggonBoy--PyreCrawl.html)
MCP server and CLI that crawls websites for AI agents: finds the answer page 2x more often than Crawl4AI, 10x cheaper per answer than letting Claude crawl.
pinkpixel-dev/pulpie-mcpMCP server that turns web pages into clean Markdown, keeping tables, code blocks, and images intact. Agents can fetch pages, save docs to your project or a global library, and crawl whole doc sites.
HasData/web-scraping-mcpMCP server for HasData's Web Scraping API: fetch any public URL through managed proxies, render JavaScript when needed, and get markdown, text, HTML or structured JSON. From 1 credit a call.
graphersal/graphersalIn-memory graph database and Gremlin traversal engine in Rust: browser playground, CLI, MCP server for AI agents, durable store, Python bindings.
rwnalds/engramSelf-hosted MCP server + dashboard giving Claude Code, Cursor & Hermes agents shared long-term memory over a git-backed markdown vault. Agent-native, Obsidian-compatible second brain — markdown RAG, n
sepehr071/awesome-iran-mcp19 open-source MCP servers (308 tools) that give AI agents the Iranian web: Digikala, Snappfood, Jabama, MrBilit, Paziresh24, Taaghche and more. Prices in Toman.
The top new MCP servers of the week, every Monday. No spam, unsubscribe anytime.