Conversational and task agents

Why it matters

The most deployed agent is not the most impressive one. It sits in a help panel or a messaging channel, answers questions about the account, looks things up, files a ticket, books a slot, and hands over to a person when it should. It touches more real users than every coding agent combined, handles their personal data, and its mistakes land on customers. Interviewers ask about it because the design is deceptively small, a prompt, a retriever, three tools, and the judgement is in what the assistant must not do and how you measure what it did.

The idea

Three kinds on one spectrum

A FAQ bot answers from a knowledge base and does nothing else. A grounded assistant retrieves from the user's own data (orders, tickets, plan) as well as the documentation, and answers questions about this account. A task agent adds a small set of write tools: create a ticket, book a slot, change an address, issue a refund under a limit. Each step up adds capability and a class of failure that did not exist before.

Grounding

Answers come from retrieved text, not from the model's memory; the Retrieval stage covers how. The retrieval set is two things: documentation (how do refunds work) and account data (where is my refund). Account reads go through a tool with the user's identity bound by the runtime, never from an identifier the model types. The prompt tells the model to answer only from what was retrieved and to say when nothing was.

A small tool set

Three to six tools is the norm: read account, search docs, create ticket, book slot, hand off. Each write tool sits behind a confirmation: the assistant states what it is about to do and waits for a yes in a form the runtime can check, a button rather than a sentence the model interprets.

{
  "name": "book_slot",
  "description": "Book a support call. Call only after the user has confirmed the time.",
  "input_schema": {
    "type": "object",
    "properties": { "slot_id": { "type": "string" } },
    "required": ["slot_id"]
  }
}

Conversation state

State is more than the message list. The runtime holds the user identity, the retrieved context for this turn, the open task (a half-booked slot), what has been confirmed, and a summary of earlier turns once the transcript grows. The model sees a rendered view of it each turn; it does not own it. That separation lets you resume on another channel, audit what the assistant knew and trim the context when it fills.

Handing off to a human

The hand-off is a tool, and the most important one. Triggers: the user asks for a person; the retriever returned nothing relevant twice; the request needs an action outside the tool set; the user is angry or distressed; the topic is on a policy list (legal, medical, account closure). It carries the summary and the retrieved facts so the human does not start from zero. Containment, the share of conversations resolved without a human, is the number the business watches, and it is easy to push up by refusing to hand off. Do not.

PII

Redact names, addresses and order details at the logging boundary, scope retrieval to the authenticated user, and never let one session's data reach another through a shared cache or a summary. Injected instructions in retrieved content (a ticket body, a review) are a path to exfiltration through a tool call; the Prompt injection lesson covers it.

Evaluation

Three numbers and a rubric. Goal completion: did the user get what they came for, graded per conversation from the transcript and the tool log. Containment rate: resolved without a human. Escalation precision: of the hand-offs, how many were needed; and its partner, escalation recall, how many conversations that needed a human never got one. Add a faithfulness check that every factual claim traces to a retrieved passage.