Guardrails: input, output and policy

Why it matters

The model is probabilistic and the prompt is advice. Every promise the product makes, no medical advice, no leaked records, no refund over the limit, has to be enforced by something deterministic around the model. That layer is guardrails. Interviewers ask about it because it is where the engineering lives: what to check, where in the request path to check it, how much latency to spend, and what to do when the check is wrong. A guard that is never wrong does not exist, so the design question is which mistakes you would rather make.

The idea

Three places to guard

An input guard sits between the user (or fetched content) and the model. An output guard sits between the model and the user. A policy guard sits between an action the model proposes and its execution. Each has a different budget and a different cost of being wrong.

Input guards. Injection and jailbreak classifiers score a message for instruction-shaped attacks. PII detection redacts before the model; the PII and data boundaries lesson covers the mechanics. Topic allow-lists keep a banking assistant on banking. Length, language and encoding checks catch the crude cases. Input guards are never complete, because injection is open-ended; they remove the obvious and raise the cost of the rest.

Output guards. Schema validation: did the output parse and conform, as in the Structured output lesson. Policy and toxicity checks against what the product may say. Citation presence: a grounded answer must cite a retrieved chunk. Grounding: a judge compares the answer to the context, which is the faithfulness metric from the Evaluation module run per request. Leak checks: no account numbers, no other tenant's names, no canary string. Link and image filters against exfiltration. Output guards turn "the model said something wrong" into "the product declined to say it".

Policy guards on actions. Which tools this user and this agent may call, which argument values are allowed (an amount under a limit, a recipient inside the tenant, a path under the root), which actions need a human. These are deterministic, written in code, evaluated before the tool runs, and they enforce the permission model from the Excessive agency lesson.

Implementation options

Rules: patterns, allow-lists, schema validators. Microseconds, free, exact and brittle. Small classifiers: a fine-tuned encoder scoring injection, toxicity or topic. Tens of milliseconds, cheap per call, needs labelled data and upkeep as language drifts. A judge model: a general model with a rubric. Hundreds of milliseconds to seconds, costly, flexible, and itself open to injection by the content it judges. Provider moderation endpoints: a hosted classifier with a fixed taxonomy. Fast, cheap, no training, no control over the categories. Most systems stack them: rules first, a classifier second, a judge only for the fraction where the blast radius justifies it.

Latency budget and placement

Inline: the guard runs before the next step; a block is immediate; its latency lands on every request. Async with rollback: the response streams while the guard runs; on a hit the client retracts the text or the action is reverted. That works for things that can be reverted, not for a sent email. Sampled: the guard runs on a percentage of traffic as monitoring, not enforcement. The rule: inline for anything irreversible or harmful when seen, async for quality, sampled for drift. Give the guard layer a number, say 150 ms of a 2 s budget, and spend it where the blast radius is largest.

False positives and the experience

A guard that blocks 2 percent of legitimate requests is a 2 percent outage for the users it hits. Measure each guard's precision and recall on a labelled set, set thresholds per risk rather than one global value, and design the block: explain what happened, offer a path, never degrade silently. Between allow and block there are soft actions: ask the model to rewrite, add a disclaimer, route to a human.

Log every block

Record the guard, the score, a hash of the input, the decision and the trace id. Blocks are how you tune thresholds, find attacks and answer appeals. A spike in blocks is either an attack or a regression, and both need a human within the hour.

Guardrails are evals run live

A grounding guard is the faithfulness eval applied to one request. A policy guard is a test case that runs in production. The Evals as release gates and runtime guardrails lesson makes the link: the same rubric runs offline on the golden set to decide a release and online on a sample to decide a response. Keep them in one codebase, so a fix to one is a fix to both.

	Rules	Small classifier	Judge model	Provider moderation
Latency	Microseconds	Tens of milliseconds	Hundreds of milliseconds to seconds	Tens to hundreds of milliseconds
Accuracy	Exact on what it matches, blind to the rest	Good on its training distribution, drifts	Best on nuance, open to injection by the content it reads	Good on its fixed categories only
Cost	None	Hosting a small model	A model call per check	Per call, cheap
Maintenance	Rule lists grow and rot	Labelled data and retraining	Rubric upkeep and eval of the judge itself	None; no control over the taxonomy
Fit	Schemas, allow-lists, formats	Injection, topic, PII at scale	Grounding, policy nuance, sampled review	Toxicity and abuse categories