Model families and reading a model card

Why it matters

Every LLM system begins with a choice no leaderboard can defend. Benchmarks rank models on tasks that are not yours, at a price and latency they do not mention. In a design review the question is never "which model is best" but "which model is good enough for this call, at this volume, inside this budget", and the honest answer is usually three models. Interviewers raise model choice early because it shows whether you think in workloads and unit costs or in brand names.

The idea

The families

Frontier closed models are served only through their vendor's API. In 2026 that means Anthropic's Claude 5.5 generation in tiers: Fable and Opus at the top for hard reasoning and long agentic work, Sonnet for most production traffic, Haiku for speed and volume. OpenAI's GPT-6 line and Google's Gemini line are organised the same way. Within a vendor the tiers share a tokeniser, a window and an API, which makes moving between them cheap.

Open-weight families publish their weights under a licence: Llama 4, DeepSeek, Qwen, GLM and Mistral. You can run them on your own hardware, fine-tune them and keep data inside your network; serving becomes your job.

Small models, a vendor's smallest tier or an open model of a few billion parameters, exist for one reason: at millions of calls a day, a model that costs a hundredth as much and answers in a fifth of the time wins every task it can do adequately. Classification, routing, extraction and reranking are those tasks.

Reading a model card

The card is the spec sheet.

• Modality: text only, or image, audio and video in; what comes out.
• Context window and max output: the current Claude lineup lists a 1M-token window with 128K max output; the GPT-6 line lists 1.05M. The output limit is the one people forget.
• Knowledge cutoff: anything after it must arrive in the prompt.
• Price per million input and output tokens: output costs several times input; across the Claude lineup, input runs from about $0.10 to $8 per million, an eighty-fold spread behind one API.
• Cache and batch discounts: prompt-cache reads at 2.5 to 10 percent of the input price; batch requests at half price.
• Thinking or effort controls: the Claude 5.5 models think adaptively and take an effort parameter; GPT-6 takes a reasoning effort from none to max. More effort means more output tokens and latency.
• Tool use and structured output support.
• Retirement date: a hosted model has a lifetime; plan the migration on the day you adopt it.
• Licence, for open weights: who may use it commercially and how.

tier:        mid          modality: text + image in, text out
window:      1M tokens    max output: 128K
price:       $x per 1M in / $y per 1M out
cache read:  5% of input  batch: 50%
controls:    effort, tools, JSON schema, streaming
retires:     announced 12 months before shutdown

Choosing by workload

Start from the call, not the model. For each call type write down how hard it is, how many arrive per day, what latency the user tolerates and what a wrong answer costs. A contract summary is hard, rare and tolerant of ten seconds; a ticket classifier is easy, constant and must answer in under a second.

The three-tier habit

Mature teams run three tiers behind one interface. A big model takes the hard reasoning: planning, multi-step agent turns, anything a human would otherwise check. A mid model takes most traffic: drafting, answering over retrieved context, ordinary tool use. A small model does classification, routing, extraction and the guard checks on every request. A router, often the small model itself, picks the tier. The pattern at least halves cost and, because the small tier answers fastest, usually improves median latency too.

Trade-offs

A frontier API gives the best quality with no operations, but you inherit a vendor's pricing, rate limits, retirement calendar and data-handling terms. Self-hosted open weights give control and a flat cost curve at volume, but you own GPUs, serving software and upgrades, plus a real gap to frontier quality on the hardest tasks. Small models are cheap and fast until the task drifts outside what they handle, and then their errors are silent. Three tiers triple the evaluation work, since every prompt change must be checked on each model. And a card is a claim, not a measurement; the only numbers that count come from your own evaluation set on your own traffic.

In an interview

Open with the workload, not the vendor: "I would split this into a small classifier, a mid model for the bulk of the chat, and a frontier model behind an explicit escalation, and I would price each tier from its card." At junior depth, know the families, the tier shape and the fields on a card. At mid depth, compute a monthly cost from calls per day, tokens per call and price, then show the batch and cache discounts changing it. At senior depth, defend a routing policy, a migration plan for a retiring model, and when self-hosting pays off. Expect "How do you know the small model is good enough?" (an evaluation set and a confidence threshold that escalates) and "What happens when the vendor retires the model?" (re-run the evaluation suite on the successor before switching).