Agentic and self-correcting RAG

Why it matters

A single-pass pipeline retrieves once and answers once. If the top chunks are off-topic, the model answers from off-topic chunks, and nothing notices. The fix that works is also the one that costs the most: let the model judge what came back and try again. Products built this way answer research questions that single-pass systems cannot, and run bills five to twenty times larger per question. Interviewers ask about agentic RAG because it is where retrieval meets agents, and because a senior engineer must be able to say when the loop pays and when it is waste.

The idea

Retrieval inside a loop

The shape is simple. Retrieve for the question. Ask the model to grade each chunk: relevant or not, on its own, with a one-word answer. If enough chunks pass, draft an answer from them. If not, ask the model to rewrite the query, often several ways at once, and retrieve again. Repeat until the grade passes or a budget runs out. Each step is one more model call, which is the whole story of its cost.

The decisions in plain words

Three published ideas are usually bundled under this name, and an interviewer will expect you to name the moves without the papers. Retrieve or not: before searching, the model decides whether the question needs the corpus at all; small talk and arithmetic do not, and skipping retrieval saves a round trip and avoids distracting chunks. Relevance grading: after searching, each chunk is judged against the question, and chunks that fail are dropped rather than passed along to confuse the answer. Corrective action: when too few chunks pass, the system does something other than answer: rewrite and retry, widen to a second corpus or a web search, or ask the user to clarify. Hallucination check: once a draft exists, a further call checks every claim against the retrieved chunks and sends unsupported drafts back round the loop or marks them with a caveat.

The bill

Count the calls for one question that loops twice. A retrieve-or-not decision. A retrieval, then grading five chunks: five small calls, or one that grades all five. A rewrite. A second retrieval and grade. A draft. A groundedness check. That is seven to twelve model calls where a single pass used one, and each later call carries the earlier chunks in its context, so a question that cost 4,000 input tokens becomes 40,000 to 80,000. Latency compounds the same way: three sequential loops at two seconds each is six seconds before the answer starts streaming, fine for a research tool and unacceptable for autocomplete.

Stop conditions and budgets

A loop without a gate is an outage waiting for a question with no answer in the corpus. Set three limits in code: a maximum number of iterations, usually two or three; a token budget for the whole question; and a wall-clock deadline. When any is hit, the system must still do something sensible: answer with an explicit caveat that the evidence was thin, hand off to a web search or a second corpus, or ask the user a clarifying question. Log which exit fired, because the ratio of budget exits to clean exits is the metric that tells you whether the corpus or the loop is the problem.

When the loop pays

The loop earns its cost where recall is the bottleneck: research questions whose first phrasing rarely matches the document, corpora with sparse or inconsistent vocabulary, and questions that genuinely span sources. It is waste for lookups. A question whose answer is one chunk, found by the first query nine times in ten, gains nothing from grading and loses on cost and latency. The Adaptive routing lesson turns this into a design: classify first, loop only for the questions that need it.

	Single pass	Grade and retry	Full agentic loop
Recall	Whatever the first query finds	Recovers from a bad first query	Highest; can widen to other sources
Latency	One retrieval plus one call	Two to three times a single pass	Five to twenty times; seconds to tens of seconds
Cost per question	One model call	Three to five calls	Five to twenty calls, each with a fuller context
Fits	Lookups, FAQs, high-recall corpora	Mixed traffic where the first query is often off	Research, analysis, low-recall or messy corpora