Function calling: schema, call, result

Why it matters

A language model on its own produces text and nothing else. Everything an agent does beyond that, from looking up an order to booking a room to editing a file, happens because the model asked for it in a form your code could act on, and your code did the work. Function calling is that form. It is the one mechanism under every agent framework and every MCP server. Interviewers ask about it because candidates who have only used a framework often believe the model runs the tool. It does not, and the whole safety and reliability story rests on knowing that.

The idea

A request to a model carries the messages so far, an optional system prompt, and a list of tool definitions. Each definition has a name, a description in plain language, and an input schema, usually JSON Schema, that says which arguments exist and what types they have.

{
  "name": "get_order",
  "description": "Fetch one order by id. Use when the user names a specific order.",
  "input_schema": {
    "type": "object",
    "properties": {
      "order_id": { "type": "string", "description": "e.g. ord_8f3k2" }
    },
    "required": ["order_id"]
  }
}

The model reads those definitions as part of its prompt. When it decides a tool would help, it does not answer in prose. It emits a tool-use block: the tool's name, a call id, and a JSON object of arguments that fits the schema. The response's stop reason changes too, from "end of turn" to "tool use", so your code can branch on a field rather than inspect text.

One lap of the loop

Your code, the runtime, finds the function registered under that name, validates the arguments, runs it, and appends two messages to the conversation: the assistant's tool-use block, then a tool-result message carrying the output and the id of the call it answers. It sends the whole conversation back. The model reads the result as a fresh observation and either answers the user or asks for another tool. The loop ends when the stop reason says "end of turn" again.

assistant  tool_use     id=call_1  name=get_order  input={"order_id":"ord_8f3k2"}
user       tool_result  id=call_1  content={"status":"shipped","eta":"2026-11-14"}
assistant  text         Your order shipped and should arrive on 14 November.

The model never executes anything. It produces a request; the runtime decides whether to honour it, under which permissions, with what timeout, and what to send back. Every guardrail you will ever put on an agent lives in that gap, between the tool-use block and the function call.

Why not parse free text

Before function calling, engineers asked the model to "reply with JSON" and parsed the output with a regular expression and hope. The model would add a preamble, drop a comma, or invent a field. Function calling fixes this at training time: models are fine-tuned on tool-use turns, and the provider's chat template frames the definitions, the call and the result with special tokens, so the model has seen this exact shape many times. Many providers also constrain decoding so the arguments must parse against the schema. You get a call you can dispatch on by name instead of a paragraph you must interpret.

Choosing, forcing, and structured output

A tool choice setting says how free the model is. Auto lets it pick between answering in text and calling any tool. Required (sometimes "any") means it must call some tool this turn. Naming one tool forces that tool. None hides the tools for a turn even though they are defined, which is useful for a final summarising step. Forcing a tool is the oldest trick for structured output: define a tool whose schema is the shape you want, force it, and read the arguments. Nothing is executed; you have used the schema machinery as a typed response format. Dedicated structured-output modes are the same mechanism under another name.

Trade-offs

Tool definitions cost tokens on every request, so forty tools with long descriptions can spend several thousand tokens before the user says a word. The model may call a tool it does not need, call with arguments that validate but are wrong, or name a tool that does not exist; the runtime must handle all three. Each lap of the loop is a full model round trip, so a task that needs six calls carries six times the latency of one. And because the model only sees what the runtime returns, a tool that returns too much, too little or something misleading corrupts every step after it.

In an interview

Describe the loop in one breath: "The request carries tool schemas, the model replies with a named call instead of text, my runtime executes it and returns a tool result, and the model continues; the model never runs anything itself." At junior depth, know the three message types and the stop reason. At mid depth, explain the tool choice modes, schema validation, and why fine-tuning plus a chat template beats parsing prose. At senior depth, talk about where policy lives (the runtime), how structured output is the same mechanism, and what the loop costs in tokens and round trips. Expect the follow-up: "What if the model calls a tool that does not exist?" The runtime returns an error result naming the valid tools and the model retries; the Parallel calls, errors and retries lesson covers the rest.