Evaluation Science and Controlled Experimentation
Add experimental rigor to repeatable agent evaluation and promotion decisions.
A rapid review of the chapter’s existing Quick Read, principles, definitions, lessons, and review material.
The chapter in one pass.
- Purpose: Add experimental rigor to repeatable agent evaluation and promotion decisions.
- Best for: AI evaluation, data, quality, product, and platform teams.
- Prerequisites: Evaluation Engineering, Trace Replay, and Run Comparison.
- Reading time: 16 minutes.
- You will learn: Dataset governance, contamination control, grader calibration, repeated trials, uncertainty, adversarial evaluation, and online experiments.
- Keep three ideas: one run is an anecdote; graders require evaluation; and promotion decisions need uncertainty and guardrails.
Reconstruct and defend this chapter’s architecture.
Reconstruct the architecture, name each boundary, and defend the tradeoffs.
Open the source exercise
Design an experiment comparing two agent configurations on 200 tasks across three risk classes. Add grader disagreement, a security hard-gate failure, cost improvement, and one segment regression. Make the promotion decision.
3. Enduring Principle
Govern evaluation datasets
Maintain source, consent or allowed use, owner, schema, task distribution, risk, difficulty, expected result, hidden checks, version, splits, retention, and known limitations. Separate development, regression, certification, adversarial, and holdout sets. Detect duplicates and benchmark contamination across prompts, skills, memory, and training inputs.
Evaluate evaluators
Deterministic checks define clear claims. Human rubrics require examples, anchors, reviewer training, blind assignment, disagreement handling, and inter-rater agreement. Model graders require versioned prompts, calibration against expert labels, position and verbosity bias tests, adversarial cases, and periodic reevaluation.
Measure variability
Run repeated trials where stochasticity matters. Report sample size, success distribution, confidence interval or uncertainty, severity, retry rate, latency, cost, and human intervention. Paired comparisons on the same tasks usually reveal differences more efficiently than unrelated aggregates.
Protect hard gates
Aggregate performance cannot compensate for unauthorized action, critical security failure, evidence fabrication, data loss, or another noncompensable condition. Segment results by workflow, repository class, risk, capability graph, and environment to avoid hiding failures.
Use a controlled promotion ladder
Progress from offline development to holdout evaluation, adversarial testing, shadow execution, limited canary, controlled comparison, and broader eligibility. Define success, noninferiority or improvement threshold, guardrails, sample, duration, stop conditions, approval, and rollback before starting.
8. Notes and lessons learned
Evaluation engineering builds the machinery to run and compare systems. Evaluation science establishes whether the comparison supports the decision being made.
9. Interview and discussion questions
- How do you detect benchmark contamination?
- When do repeated trials matter?
- How do you calibrate a model grader?
- Why can aggregate improvement hide unacceptable regression?
- What belongs in a predeclared experiment plan?
Review this chapter.
Challenge a claim, boundary, missing failure mode, unclear term, or unsupported evidence statement.
- Claim
- Boundary
- Failure
- Evidence