Evals before agents: the unglamorous habit that decides whether your AI ships

In our note on why 40% of agentic projects get canceled, the most common technical killer was the missing evaluation harness — the agent that dazzled in a demo and then drifted, unfalsifiably, in production. "Evals before agents" is our standing antidote, and it's easy to nod along to. This note is the practical version: what the phrase actually means, what the smallest worthwhile harness looks like, and why it changes the business conversation as much as the engineering one.

01Why the demo lies to you

Every demo is a biased sample. It runs the inputs someone chose because they work, in a week when the prompt was freshly tuned, on a model version that will be deprecated within the quarter. Production is the opposite: real inputs are ambiguous, incomplete, occasionally hostile, and arrive forever. Meanwhile the system underneath you moves — model updates, prompt "improvements," a new tool in the chain. Without a fixed measuring stick, you cannot distinguish improvement from regression, and every quality argument becomes two people trading anecdotes. An unmeasured agent is unfalsifiable, and unfalsifiable systems lose arguments with CFOs.

02Start with a golden set — thirty real cases beat three hundred synthetic ones

The foundation is embarrassingly simple: a versioned collection of real cases with known-good outcomes. Real tickets, real documents, real encounters — pulled from the actual workflow, not invented to look like it. Thirty carefully chosen cases that include the ugly ones (missing data, contradictory instructions, the customer who writes in two languages) will catch more real-world failure than hundreds of synthetic examples that all share the same tidy shape. Label what "acceptable" means for each — not the one perfect answer, but the boundaries of a good one.

03Score the task, not the vibes

Metrics must live at the level of the task, not the prose. Where the output is checkable, check it mechanically: did the extracted code match, did the number reconcile, did the required fields appear. Where judgment is unavoidable, write a rubric a domain expert would sign, and if you use a model as the judge, calibrate it against human graders first and keep auditing the disagreement rate. The point isn't statistical purity — it's a number the whole team accepts as the score, so "did it get better?" stops being a meeting and becomes a lookup.

04The regression gate: no green, no ship

A golden set you run occasionally is a museum. The harness earns its keep as a gate: every change — prompt, model version, retrieval tweak, new tool — runs the suite, and the score lands next to the change that produced it. A dip blocks the release until someone explains it. This is just CI culture applied to model behavior, and it produces the artifact executives actually trust: a history of scores over time, tied to decisions. When procurement or compliance asks "how do you know it still works?", the answer is a chart, not a promise.

05Evals are how you draw the autonomy boundary

Tiered autonomy — automate the routine majority, escalate the rest — is the pattern behind most agent success stories. But the boundary between tiers has to come from somewhere. The eval harness is the honest surveyor: where scores are consistently above the bar, the agent acts; where they aren't, it drafts and a human decides; the boundary moves only when the numbers say it's safe. Without evals, autonomy is set by optimism. With them, it's set by evidence — and it expands over time instead of contracting after the first incident.

06Keep it small enough to exist

The failure mode of eval advice is aspiration: a measurement platform so elaborate it's still a roadmap item when the project dies. The minimum viable harness is a folder of cases, a runner, and a score that prints in the pull request. That's a week-one deliverable, not a quarter-long initiative — it's the first thing we build in every pilot, before the agent has a name. And it has a diagnostic side effect: if a team can't define the eval, they've discovered the problem isn't specified yet — which is a "don't build this yet," delivered in week one instead of month six.

—The uncomfortable summary

Evals feel like overhead until the first time the score catches a regression your demo never would have surfaced — usually within weeks. They are also the cheapest trust-builder in the field: a scored pilot survives budget reviews, procurement questionnaires, and leadership changes that kill vibes-based projects. Discipline before enthusiasm, measurement before autonomy. It is not glamorous. It is why things ship.