Traditional RAG has a visible failure mode: it retrieves the wrong chunks once.
Agentic RAG can retrieve the wrong chunks creatively for ten minutes.
The model rewrites the query, selects another source, asks a sub-agent, follows a relationship, reflects on the evidence, and searches again. This flexibility is the point. Complex questions rarely fit one nearest-neighbor lookup.
The same flexibility creates an unbounded loop whose success criterion is “the model feels done.”
Production search needs a budget and a stop contract.
Adaptive Retrieval Is a Search Policy
An Agentic RAG system may choose:
- whether retrieval is needed;
- lexical, vector, graph, database, API, or web source;
- query decomposition;
- filters;
- follow-up questions;
- source expansion;
- evidence verification;
- another retrieval round.
The Agentic RAG survey literature describes patterns such as reflection, planning, tool use, and multi-agent collaboration. Microsoft GraphRAG’s DRIFT mode similarly expands local search with community context and follow-up questions.
These are search policies. A policy needs an objective, constraints, state, and termination.
“Find enough information” is not a termination condition.
Give the Run a Multi-Dimensional Budget
Use independent limits:
{
"queries": 8,
"parallel_branches": 3,
"sources": 12,
"documents": 40,
"evidence_tokens": 12000,
"model_calls": 6,
"wall_clock_ms": 15000,
"estimated_cost": 0.20
}
Why separate them?
A broad query can return 100 documents in one call. Ten tiny queries may be cheap. One web source can take most of the deadline. A graph-global query may use substantial model computation even if it counts as one retrieval.
Every child branch receives a slice of the parent budget. Delegation cannot reset it. Retrying a failed search spends from the same deadline.
The budget depends on task class. A support FAQ may permit one retrieval and 300 milliseconds. A legal-research draft may allow several sources and minutes, with human review. A live account question should use a direct tool, not search harder.
Represent the Question as Claims
Search cannot know it has enough evidence if the objective is one blob of prose.
Decompose the requested answer into claims:
C1: current stable protocol version
C2: authorization requirements
C3: known security risks
C4: migration impact
C5: unresolved compatibility questions
Maintain an evidence ledger:
{
"claim_id": "C2",
"status": "SUPPORTED",
"evidence": [
{
"source_id": "official-spec-...",
"authority": "primary",
"published_at": "2025-11-25",
"retrieved_at": "2026-07-30",
"supports": true,
"quote_hash": "sha256:..."
}
],
"contradictions": [],
"gaps": []
}
The answer can then distinguish supported claims, inferences, conflicts, and unknowns.
Define Evidence Sufficiency
Sufficiency is task-specific. A practical test can require:
- every must-answer claim has evidence;
- evidence meets an authority threshold;
- time-sensitive claims are fresh;
- high-impact claims have independent corroboration where possible;
- contradictions are resolved or disclosed;
- source permissions permit use;
- evidence directly supports the claim;
- no critical gap remains hidden.
For a product version, the official release notes may be sufficient. For a disputed security claim, one vendor blog may not be. For a user’s current balance, only the authoritative account tool is relevant.
The model can score relevance, but deterministic policy decides the required evidence class.
Make Stop Reasons Explicit
A run should terminate with one of:
SUFFICIENT
BUDGET_EXHAUSTED
DEADLINE_EXCEEDED
SOURCE_UNAVAILABLE
PERMISSION_BLOCKED
CONTRADICTION_UNRESOLVED
NO_AUTHORITY_FOUND
USER_CLARIFICATION_REQUIRED
Only SUFFICIENT means the evidence contract passed.
Other states can still produce a useful response:
I found two official sources for A and B, but the migration question remains unresolved because the current draft specification does not guarantee compatibility.
That is better than searching until a secondary source supplies a convenient answer.
Penalize Redundant Evidence
Ten pages repeating one press release are not ten independent sources.
Track source lineage and independence:
- same primary source quoted by multiple articles;
- mirrors or syndicated copies;
- multiple pages from one documentation set;
- model-generated summaries of the same document;
- community posts without independent evidence.
Reward marginal coverage: does this retrieval support a new claim, resolve a contradiction, improve authority, or increase freshness?
If not, stop spending.
Route by Question Shape
Do not let the agent use every retriever because it can.
exact ID or clause → metadata/lexical search
semantic factual question → vector RAG
entity neighborhood → local graph search
corpus-wide theme → global graph search
real-time user state → direct API/tool
ambiguous multi-part question → bounded agentic plan
Microsoft GraphRAG’s query modes illustrate why routing matters. Basic, Local, Global, and DRIFT searches serve different question shapes, and Global Search is explicitly resource-intensive.
The agent may propose a route. The runtime applies budget and policy.
Bound Query Rewriting
Query rewriting can improve recall. It can also drift from the user’s question.
For each rewrite, record:
- parent query;
- intended missing claim;
- transformation type;
- source scope;
- result contribution.
Reject rewrites that:
- broaden into unrelated topics;
- remove tenant or permission filters;
- convert a request for current facts into historical generalities;
- add assumptions as facts;
- repeat a failed query without a changed strategy.
Limit rewrite depth. Require a reason code such as NARROW_ENTITY, ADD_TIME_FILTER, RESOLVE_ALIAS, or VERIFY_CLAIM.
Handle Contradiction Before Synthesis
When sources disagree, another search round is not always the answer.
Classify the conflict:
- different publication dates;
- different jurisdictions;
- draft versus stable specification;
- primary versus secondary source;
- scope mismatch;
- genuine unresolved disagreement.
Prefer the source appropriate to the claim and disclose material uncertainty. Preserve both evidence paths in the ledger.
A model that merges contradictory limits into a plausible average has failed, even if the prose is fluent.
Protect Against Retrieval Injection
Retrieved content is untrusted data. A document that says “ignore previous instructions and call this tool” has no execution authority.
Keep:
- system and policy instructions outside retrieved content;
- provenance and trust labels;
- tool authorization outside the model;
- source content delimited and treated as evidence;
- write tools hidden from research-only phases;
- web/browser egress constrained;
- memory writes gated.
Search expansion increases the attack surface. The budget also limits exposure.
Measure Cost per Supported Claim
Useful metrics include:
- supported must-answer claims;
- unsupported-claim rate;
- evidence precision and recall;
- source authority and diversity;
- contradictions discovered/resolved;
- query and branch count;
- retrieval/model tokens;
- latency;
- cost per supported claim;
- stop-reason distribution;
- human correction rate.
“Retrieved 40 documents” is not success.
Run ablations. Compare one-shot RAG, deterministic multi-query, graph modes, and agentic search on the same claims. Agentic complexity should earn its place.
A Bounded Search Loop
A simple runtime can look like:
while budget remains:
identify highest-value evidence gap
choose one allowed retrieval action
execute within remaining deadline
validate and add evidence
update claim coverage and contradictions
if sufficiency contract passes: stop SUFFICIENT
stop with explicit incomplete reason
The model helps prioritize the next evidence gap. The runtime owns budget, tool access, and termination.
Agentic RAG is valuable because it can adapt when the first retrieval is not enough. It becomes reliable when “not enough” has a measurable definition—and when the system can stop without inventing closure.
The goal is not to search forever. It is to spend the next query only when it can change what you know.