The 45-minute AI system design interview

Why it matters

The AI system design round is forty-five minutes, one prompt, and an interviewer who has heard that prompt answered thirty times. They are not waiting for a clever architecture. They are listening for whether you scope before you draw, whether you can say how you would know the thing works, and whether you give numbers unasked. Most candidates run out of time in the design because they started it at minute two. The structure here is a clock: each band has a fixed length and must produce one thing before the next begins.

The idea

Six bands. A band that overruns steals from the deep dives, and the deep dives are where the hiring decision is made.

The six bands

0 to 5, clarify and scope. Users, data and its freshness, volume, acceptable latency, budget per request, and what good looks like in one sentence. Produce: a requirement list with numbers.

5 to 10, the eval-first framing. Before a box is drawn: the metric, the golden set, the threshold that gates release and the guardrail that must never trip. Produce: one eval statement under the requirements.

10 to 20, high-level design. The pipeline or the loop, the components, the data flow from input to answer, offline work beside the online path. Produce: a drawing the interviewer can point at.

20 to 35, deep dives. The interviewer picks two or three boxes: retrieval, the agent loop, the evals, cost, safety. Go one level down in each: mechanism, number, failure. Produce: a decision per dive with its reason.

35 to 42, trade-offs and failure modes. What you gave up, what breaks first, what happens when the model is wrong, the tool fails or the index is stale. Produce: ranked risks with a mitigation each.

42 to 45, ship and measure. The first slice, the first number you would watch, and what would change the design. Produce: a plan for week one.

What interviewers listen for

In scoping, questions that change the design: "how fresh must answers be?" changes the architecture; "which cloud?" does not. In the eval band, whether you ask for the measurement before they do. In the design, data flow: what crosses each arrow and roughly how many tokens. In deep dives, mechanism and numbers that hang together: a 3-second budget with 2 seconds of generation leaves one second for everything else, and you should be the one to notice. In trade-offs, failure modes named unprompted. In the last band, restraint: a first slice that is small and measurable.

Three habits

Strong candidates show three habits in the first ten minutes. They ask for the eval before the architecture: "what does a good answer look like, and how will we score it?" They name failure modes before being asked: "the retriever will miss, the model will fill the gap, and the user will see a confident wrong answer with a citation; here is where I catch that." They give numbers: requests a day, tokens a request, cost a day, milliseconds a stage. A number wrong by a factor of two still shows you think in quantities. No number shows nothing.

The whiteboard

Left column: requirements and the eval statement, never erased. Centre: the pipeline, left to right, offline above online. Right column: a running list of risks and open questions, which becomes the trade-offs band for free. Label arrows with what flows and how much, and draw boxes big enough to point at later.

Common mistakes

Jumping to a framework: "an agent framework and a vector database" is a shopping list, and it suggests you cannot describe the mechanism underneath. No numbers: a design with no volume, tokens or cost cannot be checked, so it cannot be credited. Ignoring cost and safety: the interviewer will raise both, and raising them first is worth more than answering well later. Over-designing: multi-agent for a lookup, fine-tuning for a formatting problem. And a design band that runs to minute thirty, leaving nothing to dive into.

	Junior	Mid	Senior
Clarify and scope	Asks who and what; takes the prompt as given	Turns answers into requests a day and a latency budget	Finds the requirement that changes the design and says why
Eval-first framing	Names a golden set and one metric when asked	States metric, golden set and threshold before drawing	Adds the guardrail and gates release on the eval
High-level design	A working naive pipeline with its boxes named	Data flow with token counts on the arrows	Offline and online paths, caching, where humans sit
Deep dives	Explains one box when pointed at	Mechanism, number and failure for each box picked	Trades across layers; changes the design when a number breaks
Trade-offs and ship	Lists risks when asked	Names failure modes unprompted with a mitigation each	Ranks risks by severity, ships a slice behind a flag, names the first number

Trade-offs

The structure is a defence against the clock, and it can sound rehearsed. Interviewers who hear the same six headings push off-script, and a candidate who can only run the script stalls. Treat the bands as what must exist by each minute, not as a speech. An impatient interviewer may read the eval band as stalling; say why you are doing it ("every later decision moves this number") and move on. And the lengths bend with the prompt: a diagnosis spends longer on evidence and has no pipeline to draw.

In an interview

Open with the structure out loud: "Five minutes on scope and numbers, five on how we would measure it, ten on the design, then deep wherever you want." Managing the clock visibly buys permission to ask questions. At junior depth, run the bands and produce the drawing and one metric. At mid depth, add the eval statement before the drawing, numbers on the arrows and failure modes unprompted. At senior depth, trade across layers in the deep dives, give a cost per request, and close with a first slice and a rollout. Expect "what would you build first?" at minute forty and "how would you know it got better?" at any minute.