Coding agents

Why it matters

Coding agents are the most used agent type in 2026. The loop is plain: read the repository, edit files, run the tests and commands, read what failed, edit again, open a pull request. What makes it work is not the model alone but the environment: code is the one domain where the agent can check its own work against ground truth on every step. Interviewers use coding agents as the worked example for every agent question, because the candidate has probably used one.

The idea

Why code is a good environment

A compiler, a type checker, a linter and a test suite are oracles that return a verdict in seconds, at no token cost and with no opinion. The agent does not need to believe its edit is right; it runs the tests. Compare a writing agent, which has no oracle at all. Version control adds undo: every change is a diff on a branch, reviewable and revertible. Those two properties, verifiability and reversibility, are why coding agents can be given long leashes.

The loop

Read: list files, grep, open the ones that matter. Plan: a short list the model writes for itself. Edit: apply a diff. Run: the test command, the build, a script. Observe: the output, truncated to what matters. Repeat until the tests pass or a budget runs out, then open a pull request with a description of what changed and why. A run of a hundred steps is normal; a thousand is not rare.

read   src/billing/invoice.ts   220 lines
edit   src/billing/invoice.ts   +14 -3
run    npm test -- invoice      FAIL 1 of 42: rounding on negative totals
edit   src/billing/invoice.ts   +2 -1
run    npm test -- invoice      PASS 42 of 42

Long-horizon state

A task across hundreds of steps and dozens of files does not fit in working memory, even a large one, once every file read and test log sits in the transcript. The harness manages it: tool results are truncated, old observations are summarised or dropped, the plan and the list of files touched live in a scratchpad the model rereads, and when the context fills the run compacts and continues. The Context engineering lesson covers the techniques; losing track of what was already changed is the main long-horizon failure.

Sandboxing and the human boundary

The agent runs commands, so it runs in a sandbox: a container or microVM with the repository mounted, no production credentials, network limited to package registries, and a permission layer that asks before destructive commands. The human review boundary is the pull request. Nothing reaches the main branch without a reviewer, CI runs on the agent's branch like anyone else's, and the PR description is the agent's account of its work. That boundary is what makes an autonomous run acceptable: the blast radius is a branch.

Evaluation

Benchmarks of real repository tasks exist: take an issue from a public project, give the agent the repository at that commit, and check whether hidden tests pass afterwards. They measure end-to-end task completion, not code quality, and teams build private versions from their own history, closed issues with known fixes, replayed. Alongside: pass rate, steps and tokens per task, how often the agent asks for help, and how often reviewers reject a passing PR.

The harness is the product

Two products on the same model differ by their harness: how context is assembled (which files, which docs, the project's conventions file), how tools are designed (an edit tool that applies diffs reliably, a test runner that returns the failing assertion rather than ten thousand lines), what the permission model asks about, and how the run is compacted. Most of the quality gap between coding agents lives here, not in the model.