Why evals decide the product

Why it matters

A unit test asserts that a function returns the same value every time. A model-backed feature does not work like that. The same prompt returns a different answer on the next call, another after the provider updates the model, and another when a retrieved chunk changes. Quality is a distribution, not a value. Teams that treat an LLM feature like code ship after looking at three examples and hoping. Interviewers know this, which is why almost every system-design question in this field ends with the same sentence: "and how would you know it got better?" The eval set is the answer, and it doubles as the specification.

The idea

Quality is a distribution

Take a support assistant and the question "can I get a refund after 30 days?". Run it a hundred times. Ninety answers are right, six are vague, four invent a policy. No single run tells you that; only the hundred do. An eval asks the question many times, across many questions, and reports a rate. Change the prompt, run it again, compare the rates. That rate is the only thing that can tell you a change helped, because for any one example you can always find a run that looks fine.

The spec and the test suite in one

With ordinary code the spec is a document and the tests check it. With a model, a prose spec is close to useless: the model does not read it and the engineers cannot check it. Writing the spec as cases, each with an input and a statement of what good looks like, gives you both at once. "Refund questions past 30 days must cite the policy page and must not promise money" is a case. Fifty such cases are the product definition, and running them is the test.

The 2026 gap

Most teams running agents in production can now see what the agent did: traces, token counts, latency, tool calls. Far fewer can say whether a change made it better. The O'Reilly 2026 stack article, citing LangChain's survey, puts it at roughly nine in ten with observability against about half with evals. Observability is about the past; evals are about the next change. The half without them change prompts on instinct and learn about regressions from customers.

What evals buy you

Three things. Safe change: swap a model, shorten a prompt or add a tool and know within minutes whether the score moved; it is the only way to take the cheaper model without fear. A number to argue with: "the new prompt is better" becomes "faithfulness went from 0.81 to 0.88 on refunds and dropped two points on billing", which ends meetings. And the interview: every design answer in this track ends with how you would measure it, and the eval set is how.

The minimum viable eval

You do not need a platform. Thirty cases in a JSON file, a script that runs each through the system and checks the output with code or a judge, and a number printed in CI. That is an afternoon. The Building a golden set lesson covers the retrieval-specific version with reference chunks; the general version needs only an input, an expected behaviour and a check.

{
  "id": "refund-31d",
  "input": "Can I return this after 31 days?",
  "expect": { "cites": "returns-policy", "must_not": ["full refund guaranteed"] },
  "check": "judge:policy-rubric-v2"
}

The flywheel

Ship, watch production, and every failure a user reports or a judge flags becomes a new case. The set grows toward the failures that happen, not the ones you imagined. Within a few months it is a map of everything that ever went wrong, and no change can bring any of it back without the number dropping.

Trade-offs

An eval set measures what is in it. A set written before launch measures the imagination of the team; one grown from production measures yesterday's users. Judges add their own noise, and a two-point move on 30 cases is not a signal. The cost is real: cases need labelling, judges cost tokens, and someone has to own the set or it rots, at which point a passing eval manufactures confidence. The deeper trap is optimising for the number: once the set is the target, a prompt that games the judge scores well and serves users badly, which is why a held-out slice and a human sample stay in the loop.

In an interview

Say it before you describe any architecture: "Before I change a prompt I want an eval set: a few dozen cases drawn from real traffic with a check for each, run in CI, so every change gets a number." At junior depth, know that outputs vary, so quality is a rate measured on a set of cases. At mid depth, describe the minimum viable eval, the flywheel from production failures, and when a code check beats a judge. At senior depth, talk about the set as the specification, the gap between observability and evals, the overfitting trap and who owns the thresholds. Expect "How big does it need to be?" (enough that the move you care about is not noise: 30 to start, growing from failures) and "What do you do before you have users?" (write cases from the spec and replace them with traced ones within weeks).