Evals that gate releases
If no test failed, you aren't testing — you're hoping.
A self-hosted evaluation control plane for AI systems. Drive the real deployed system, score behavior in layers, and turn the verdict into a CI exit code that can block a release.
What you get
An evaluation control plane, not another metrics tab
PeakEval treats evals as infrastructure: reproducible runs, layered graders, and a verdict your pipeline can act on.
Drive the real system
Run evals against your actual deployed stack over HTTP, not a mocked model. What you score is what ships.
Score behavior in layers
Grounding, safety, tool use, and policy each get their own graders so a single number never hides a real regression.
A gate, not a dashboard
Every run resolves to a verdict and a CI exit code. Wire it into your pipeline and let failing evals block the release.
Self-hosted by default
Runs entirely in your infrastructure. Prompts, traces, and scores never leave your network unless you choose.
Model-agnostic
Bring any provider or a local model. Graders are pluggable, so you decide what counts as pass and what counts as fail.
Traceable verdicts
Every score links back to the exact input, output, and grader reasoning. The score is a fact you can audit.
How it works
From a YAML file to a release gate
Three steps take you from describing behavior to blocking a bad deploy — no dashboards to babysit.
Define suites
Describe the behavior that matters in YAML: the inputs to send, the graders to apply, and the thresholds that count as a pass.
# suites/checkout-agent.yaml
target: https://api.internal/agent
checks:
- grounding.no_hallucinated_price
- policy.no_refund_without_order_idRun against the real system
PeakEval drives your deployed endpoint, captures every trace, and scores each layer independently so failures point at a cause.
$ peakeval run suites/checkout-agent.yaml
scoring 4 checks over 120 cases...
grounding 0.98
policy 0.71 ← below thresholdGate the release
The run resolves to a single verdict and an exit code. Drop it into CI and a failing eval blocks the deploy automatically.
# .github/workflows/deploy.yml
- run: peakeval run --gate release
# exit 1 → deploy is blockedPricing
Start free. Pay when your team scales.
The engine is open source and self-hosted. Paid tiers add collaboration, controls, and support.
Open Source
The full evaluation engine, run entirely in your own infrastructure.
- Unlimited suites & runs
- Layered graders
- CI exit-code gating
- Community support
Team
PopularShared runs, history, and access controls for teams shipping AI to production.
- Everything in Open Source
- Shared run history & diffs
- Role-based access control
- Slack & PagerDuty alerts
- Priority support
Enterprise
Deployment support, SSO, and guarantees for regulated environments.
- Everything in Team
- SSO / SAML & audit logs
- Air-gapped deployment
- Dedicated solutions engineer
- Custom SLAs
FAQ
Questions, answered
Still unsure? Read the docs or reach out — the score is a fact, and we are happy to show you how it is computed.
What does self-hosted actually mean here?
PeakEval runs entirely inside your own infrastructure. Your prompts, model outputs, traces, and scores stay on your network — nothing is sent to a third party unless you explicitly configure it.
Which models and providers are supported?
PeakEval is model-agnostic. It drives your deployed system over HTTP, so it works with any hosted provider, a self-hosted open model, or a multi-model pipeline. Graders are pluggable too.
How does it block a release?
Every run resolves to a single verdict and a process exit code. Add `peakeval run --gate release` to your CI pipeline and a failing eval returns a non-zero exit, which blocks the deploy like any other failing check.
Is this a replacement for observability tools?
No. Observability tells you what happened in production. PeakEval decides whether a change is allowed to ship in the first place — it is a gate, not a dashboard.
How is a check scored as pass or fail?
You define graders per layer — grounding, safety, tool use, policy — each with its own threshold. A run only passes when every check clears its threshold, so one number can never hide a real regression.