Archive/AGENTS

AI agent failure modes: a field guide and postmortem template

Wrong arguments, irreversible actions, runaway loops and confident false success. How agents fail, how to investigate it, and which guardrail stops each failure.

AI agent failure modes: a field guide and postmortem template

An AI agent is a model running in a loop: it reads the situation, chooses a tool, observes the result and decides what to do next. That loop is what makes agents useful, and it is also what makes their failures different from a chatbot's. A chatbot that hallucinates produces a wrong paragraph. An agent that hallucinates can book a non-refundable trip, delete a branch or email a customer.

This field guide catalogues the failure modes that show up most often when agents meet real systems, explains how to investigate an incident, and maps each failure to the guardrail that prevents it. It ends with a postmortem template you can copy.

Why agents fail differently

Three properties make agent failures harder than single-call LLM failures:

Before building an agent at all, it is worth asking whether a fixed workflow would do; our agent-or-workflow decision guide helps with that call.

The eight most common agent failure modes

1. Wrong arguments to the right tool

The agent picks the correct tool but passes a wrong value: the wrong date format, a customer ID from an earlier turn, a currency in the wrong unit. These failures are common because tool schemas are often ambiguous. A parameter called date does not say whether it means check-in or booking date, or which timezone.

2. Plausible but invented inputs

When information is missing, agents tend to fill the gap with a reasonable guess rather than asking. An invented email address, an assumed budget or a guessed file path all look fine in a log until someone notices the guess was wrong.

3. Irreversible actions taken too early

The agent completes a purchase, sends a message or deletes data before it has finished gathering information, often because the task description rewarded "getting it done."

4. Runaway loops

The agent repeats the same failing tool call, oscillates between two plans, or keeps searching for something that does not exist. Without limits, loops burn money and API quotas. See step budgets and stopping rules for the fix.

5. Misread tool results

The tool returns an error, an empty list or a truncated page, and the agent treats it as success. Error messages written for humans ("Something went wrong") are a frequent cause.

6. Goal drift

Over a long run the agent loses track of the original constraint, for example a budget limit stated in the first message, and optimises for a proxy such as "found an available option."

7. Prompt injection through tool outputs

Web pages, emails and documents the agent reads can contain instructions. A model that treats retrieved text as commands can be steered into leaking data or taking actions the user never asked for. This is listed first in the OWASP Top 10 for LLM applications; our guide on red-teaming an LLM chatbot shows how to test for it.

8. False success reports

The agent reports "Done! Your meeting is booked" when the final call failed or never happened. This is the most damaging failure because it hides all the others.

Guardrails mapped to failures

Which guardrail stops which failure
Failure modePrimary guardrailSecondary guardrail
Wrong argumentsStrict schemas with enums, units and examplesValidate arguments before execution
Invented inputsRequire a source for every argumentAsk-the-user tool with clear triggers
Irreversible actions too earlyHuman approval for side-effecting callsDry-run or draft mode first
Runaway loopsStep, time and cost budgetsDetect repeated identical calls
Misread tool resultsStructured, machine-readable errorsPost-condition checks in code
Goal driftRestate constraints in each step's contextFinal check against original request
Prompt injectionTreat tool output as data, not instructionsLeast-privilege tool permissions
False successVerify outcomes in code, not by model reportShow evidence (IDs, receipts) to user

Two principles run through the table. First, move checks out of the model and into code wherever possible: schema validation, budgets and outcome verification do not get tired or persuaded. Second, put a human on the irreversible edge. Patterns for doing that without annoying people are in human-in-the-loop patterns.

Tool design matters as much as the model. Clear names, narrow scopes and explicit error messages prevent the first, fifth and eighth failure modes; see designing tools agents can actually use.

How to investigate an agent incident

When something goes wrong, resist the urge to tweak the prompt immediately. Reconstruct first.

  1. Collect the full trace: system prompt, user messages, every tool call with arguments, every tool result, and timestamps.
  2. Find the first wrong step. Not the step where the damage happened, but the earliest decision that was incorrect given what the agent knew.
  3. Classify the failure using the list above. Most incidents are one primary mode plus a missing guardrail.
  4. Ask why the guardrail did not catch it. Was there no check, or was the check bypassable?
  5. Turn the incident into an eval case. Every real failure should become a regression test; see evaluating agents end to end.

Monitoring agents in production

Guardrails prevent known failures; monitoring finds the ones you did not anticipate. A small set of signals catches most problems early:

Sample a handful of full traces every week and read them end to end, even when the metrics look healthy. Traces reveal slow problems such as gradually longer plans, unnecessary tool calls and polite but useless clarifying questions that never trigger an alert.

A blameless postmortem template

textagent-postmortem.md
Title: <one line, what happened, not who>
Impact: <users affected, money, data, time to detect, time to resolve>
Timeline: <timestamps of key steps from the trace>
First wrong step: <step number, what the agent did, what it should have done>
Failure mode(s): <from the field guide>
Why guardrails missed it: <missing check / bypassed check / wrong threshold>
Fixes:
  - Prevent: <code-level guardrail>
  - Detect: <alert or post-condition>
  - Test: <new eval case ID>
Open questions: <what we still do not understand>

Write postmortems about the system, not the model's "personality." "The model was overconfident" is not actionable. "The booking tool accepted a non-refundable fare without an approval step" is.

Frequently asked questions

What are the most common AI agent failure modes?

The most frequent are wrong tool arguments, invented inputs when information is missing, irreversible actions taken too early, runaway loops, misreading errors as success, losing track of constraints, prompt injection through retrieved content, and false reports of success.

How do I stop an AI agent from taking dangerous actions?

Give it least-privilege tools, require human approval for anything irreversible or expensive, validate arguments in code before execution, and verify outcomes independently instead of trusting the agent's own report.

Why do agents get stuck in loops?

Usually because a tool keeps returning an error the model cannot interpret, or because the stopping condition is vague. Set hard budgets on steps, time and cost, detect repeated identical calls, and return structured errors that say what to do next.

Should every agent failure become a test case?

Yes. Converting real incidents into regression tests is the fastest way to build an agent eval suite that reflects how your system actually breaks.

RK
WRITTEN BYRafael KimAgents and tooling. Has personally been booked into a yurt by a language model.

Keep reading

Liked this? The Sunday Diff has one like it every week.