Skip to download
forensi

Forensi · Field guide · 2026

10 ways RAG agents fail

and how to spot each one in a trace.

A field guide for engineers running RAG agents in production. Ten failure modes, each with the trace signal that gives it away. All case data is illustrative.

Free · 9 pages · no email required

The ten

Failure modes, and where they show up in a trace.

  1. 01

    Stale source wins

    Two in-context chunks cover the same topic with different dates, and the citation points to the older one.

  2. 02

    Retrieved is not used

    The chunk that holds the answer is in the candidate list, just past top_k, and never reached the model.

  3. 03

    Sources conflict, no caveat

    Two chunks in context give different values for the same field, and the answer mentions only one.

  4. 04

    Authority inversion

    An informal source is cited while an official one on the same topic is also in context. Ranking used similarity only.

  5. 05

    Tool error swallowed

    Span status is OK, but the payload is an error, empty, or null — and the answer invents facts.

  6. 06

    Chunk split at the wrong line

    The cited chunk ends mid-section, and the next chunk starts with except, unless, however, or a table continuation.

  7. 07

    Missing metadata filter

    Retrieval has no filter, or the docs’ region, tenant, or product does not match the user’s context.

  8. 08

    Right tool, wrong arguments

    The tool succeeded with a valid payload for a different id, date, or entity than the one the user asked about.

  9. 09

    Context overflow

    Prompt tokens near the limit, truncation flags, or the answer-bearing chunk buried in a long context.

  10. 10

    Injected instructions in a document

    Imperative text inside retrieved content, followed by a behaviour change or a tool call the user did not request.

Running agents in production? Send 20–50 failed runs and get a free root-cause report.