Forensi · Field guide · 2026
10 ways RAG agents fail
and how to spot each one in a trace.
A field guide for engineers running RAG agents in production. Ten failure modes, each with the trace signal that gives it away. All case data is illustrative.
Free · 9 pages · no email required
The ten
Failure modes, and where they show up in a trace.
- 01
Stale source wins
Two in-context chunks cover the same topic with different dates, and the citation points to the older one.
- 02
Retrieved is not used
The chunk that holds the answer is in the candidate list, just past top_k, and never reached the model.
- 03
Sources conflict, no caveat
Two chunks in context give different values for the same field, and the answer mentions only one.
- 04
Authority inversion
An informal source is cited while an official one on the same topic is also in context. Ranking used similarity only.
- 05
Tool error swallowed
Span status is OK, but the payload is an error, empty, or null — and the answer invents facts.
- 06
Chunk split at the wrong line
The cited chunk ends mid-section, and the next chunk starts with except, unless, however, or a table continuation.
- 07
Missing metadata filter
Retrieval has no filter, or the docs’ region, tenant, or product does not match the user’s context.
- 08
Right tool, wrong arguments
The tool succeeded with a valid payload for a different id, date, or entity than the one the user asked about.
- 09
Context overflow
Prompt tokens near the limit, truncation flags, or the answer-bearing chunk buried in a long context.
- 10
Injected instructions in a document
Imperative text inside retrieved content, followed by a behaviour change or a tool call the user did not request.
Running agents in production? Send 20–50 failed runs and get a free root-cause report.