- CollectionLLM Gateway & Routing
- Classificationagent-native
- LicenseAGPL-3.0 · v0.1.5 (March 2026)
- MCPPlanned
- Agent SkillUnavailable
- InterfacesCLI · http
What it does
Routerly is a self-hosted LLM gateway with one distinguishing primitive: an LLM-native routing policy that uses a small LLM to make per-request routing decisions, alongside conventional rule-based policies (cost, latency, fallback, sticky session). The other policies are familiar — budgets, tenant isolation, cost tracking — but the operational story is unusual: no database, no Redis, just a JSON config file. That makes it interesting for small teams running a single agent fleet who want intelligent routing without operating a stateful gateway cluster.
It is drop-in compatible with the OpenAI and Anthropic SDKs (just swap the base URL), and supports OpenAI, Anthropic, Gemini, Mistral, and local Ollama.
Agent-facing interfaces
- Catalog protocols: CLI, http
- MCP: ⚠️ Not yet published. Routerly fronts LLM endpoints; an MCP-on-MCP routing surface is on the roadmap.
- Agent Skill: ⚠️ Not yet published.
- Onboarding type:
cli
How agents onboard
# Run the gateway:
docker run -p 8080:8080 -v ./routerly.json:/config/routerly.json inebrio/routerly:latest
# Point any OpenAI-compatible client at it:
export OPENAI_BASE_URL=http://localhost:8080/v1
export OPENAI_API_KEY=<your tenant key>
See the full onboarding section in the dossier.
Protocol surface
| Interface | Detail |
|---|---|
| OpenAI-compatible HTTP | /v1/chat/completions, /v1/completions, /v1/embeddings |
| Anthropic-compatible HTTP | /v1/messages |
| JSON Config | Single file — providers, tenants, policies, budgets |
| Docker / single binary | Self-host anywhere |