Generative AI in one page

Why it matters

Half the bad decisions in LLM products come from treating the model as something it is not: a database that ought to know a fact, a function that ought to return the same answer twice, a colleague who checked before speaking. The other half come from a vocabulary gap, where a team argues about "context" and "memory" and means different things. Interviewers test the vocabulary in the first ten minutes, often sideways: a candidate who says "the model remembers the user" has already said plenty.

The idea

A generative model produces new content, text, code, images or audio, by predicting it piece by piece conditioned on a prompt. For a language model the piece is a token, and the whole mechanism is one loop: read the prompt, predict a probability for every token in the vocabulary, sample one, append it, repeat until a stop token. Everything else in this track is scaffolding around that loop.

Three things that changed since 2022

Three changes turned language models from research artefacts into components.

Scale. Larger models trained on far more text became general: one model handles summarising, translating, coding and planning without per-task training. Scale also bought longer context, from a few thousand tokens to a million.

Instruction tuning and RLHF. A pre-trained model completes text; ask it a question and it may continue with more questions. Post-training on instruction examples, then on human preferences, turned completion into conversation and gave the model a notion of being helpful, honest and harmless.

Tool use. Models were trained to emit structured calls rather than only prose. A completion can now be an action: look this up, run this code, book this slot. That one change is what makes retrieval pipelines and agents possible.

Model, product, system

A model is weights plus a tokeniser: a function from a token sequence to a distribution over the next token. A product is a model behind an interface, with a system prompt, a conversation and a user. A system is the product plus retrieval, tools, memory, evals, guardrails and a cost model. Interviews are about systems. Answering at the model level, "use a better model", is the commonest way to lose a design round: every candidate can pick the same models, and the interviewer wants what you build around one.

The nine words

Token: the unit the model reads and writes, usually a subword. Context window: the maximum tokens the model can attend to in one call, prompt and completion together. Prompt: everything you send, system instructions, conversation and any retrieved text. Completion: what comes back, text or a tool call. Embedding: a vector that places a piece of text so that similar meaning lands nearby. Retrieval: finding the chunks relevant to a query, usually by embedding similarity, and writing them into the prompt. Tool: a function the model can ask to have called, described by a schema. Agent: a loop in which the model chooses tools and acts on the results until a stop condition. Eval: a repeatable measurement of whether the system's outputs are good.

What it is not

Not a database. The model's knowledge is a compressed statistical trace of its training text. There is no row to look up, no update statement, no freshness date. Facts that changed after training are gone; rare facts were never reliably there.

Not deterministic. Output is sampled. Low temperature narrows the choice but does not remove variance, and the same prompt can return different completions across calls, versions and providers. Anything that depends on the output must tolerate that.

Not grounded by default. Nothing ties the completion to a source unless you put the source in the prompt and check the output against it. A fluent answer and a true answer look identical from the outside.

Each "not" is why a module exists: retrieval because it is not a database, evals and structured output because it is not deterministic, citations and guardrails because it is not grounded.

	Classic software	ML model (classifier)	Generative model
Determinism	Same input, same output	Same input, same output; behaviour fixed at training	Sampled; same input can differ across calls and versions
Testing	Unit tests on exact values	Accuracy on a held-out set	Evals with judges and golden sets; no exact expected string
Cost per call	Microseconds, near free	Milliseconds, cheap	Cents and seconds, priced per token
Failure mode	Throws or returns wrong type	Wrong label, usually with a confidence score	Fluent, confident, wrong, with no signal that it is

Trade-offs

What you gain is a general component that handles inputs nobody wrote code for. What you pay is cost per call in cents rather than microseconds, latency in seconds, outputs that must be verified, and behaviour that shifts when the provider updates the model under you. The honest framing is a trade of engineering effort for operating cost plus verification effort. For a fixed, enumerable set of inputs a classifier or a rule is still the better tool, and saying so earns points.

In an interview

The sentence: "A language model is a next-token predictor; the product is what I build around it to make it grounded, bounded and measured." At junior depth, define the nine words and the three nots. At mid depth, explain the loop and show how each not implies a component: retrieval, structured output, evals, guardrails. At senior depth, tie the nots to the economics: what grounding costs in tokens, what verification costs in latency, when a model is the wrong tool. Expect "why does it hallucinate?" The answer is one sentence: it predicts plausible text rather than retrieving a fact, so the fix is grounding and checking, never instructing it to be accurate.