- CollectionObservability & Tracing
- Classificationagent-native
- LicenseOpen-source (see repository)
- MCPAvailable
- Agent SkillUnavailable
- InterfacesMCP · REST · GRPC · CLI · http · API
What it does
AgentEvals scores AI agent behavior from OpenTelemetry traces without re-executing the agent. It ingests OTLP or Jaeger JSON, compares runs against golden eval sets (expected tool trajectories, responses), and reports pass/fail metrics suitable for CI. A bundled web UI and OTLP receiver support live streaming during development; Kubernetes Helm chart and Docker image support production-adjacent deployment next to agent workloads.
Agent-facing interfaces
- Interaction pattern: CLI + OpenTelemetry + optional MCP + live OTLP receiver
- Catalog protocols: MCP, REST, GRPC, CLI, http, API
- MCP: ✅ Available (ships with agentevals-cli[live])
- Agent Skill: ⚠️ No standalone npx skills add … pack; the upstream repo documents Claude Code slash workflows under .claude/skills/ for repository contributors, not a published ClawHub skill.
- Onboarding type:
mcp
How agents onboard
Interaction pattern: CLI + OpenTelemetry + optional MCP + live OTLP receiver
pip install agentevals-cli
See the full onboarding section in the dossier.
Protocol surface
| Interface | Detail |
|---|---|
| CLI | agentevals run, agentevals serve, agentevals mcp, agentevals evaluator list |
| OTLP | HTTP receiver on 4318 (and gRPC 4317 patterns via Collector) — see upstream README |
| REST API | Embedded in serve — /docs, /redoc |
| MCP | Port 8080 in containerized deployment — README |
| Kubernetes | Helm chart in charts/agentevals/ |