- CollectionBrowser & Web Execution
- Classificationagent-native
- MCPAvailable
- Agent SkillUnknown
- InterfacesMCP · http
What it does
Crawl4AI is an open-source web crawler and scraper built for LLMs and AI agents: JavaScript rendering, LLM-friendly markdown output, extraction strategies, anti-bot hooks, shadow DOM handling, and crash recovery. Agents use it to turn arbitrary URLs into context-window-sized, structured text instead of raw HTML soup.
Agent-facing interfaces
- Interaction pattern: Python library + Docker + MCP
- Catalog protocols: MCP, http
- MCP: ✅ MCP server shipped in ecosystem (see repo and “Crawl4AI MCP Server” documentation in community/dev posts linked from project).
- Agent Skill: ⚠️ No single official npx skills add unclecode/… skill pack required by upstream.
- Onboarding type:
mcp
How agents onboard
Interaction pattern: Python library + Docker + MCP
See the full onboarding section in the dossier.
Protocol surface
| Interface | Detail |
|---|---|
| Python | Primary library |
| Docker | Recommended for MCP / full browser stack |
| MCP | Model Context Protocol integration |
| HTTP | When deployed as a service |