"Keep a human in the loop" is the standard answer to AI risk. It is good advice, badly implemented more often than not. An agent that asks for approval on every step trains its human to click "Approve" without reading. An agent that never asks takes the one action that should have been checked. Both failures are design problems.
This guide covers human-in-the-loop (HITL) patterns for AI agents and LLM workflows: where to put human checkpoints, how to decide what needs approval, how to design review screens people actually read, and how to avoid approval fatigue.
What human oversight is for
Human checkpoints serve four distinct purposes, and each suggests a different design:
- Preventing harm from irreversible or high-impact actions: payments, deletions, external communication.
- Supplying information the agent does not have: preferences, missing details, judgement calls.
- Catching quality problems before outputs reach customers.
- Creating accountability: a record of who approved what.
Be clear about which purpose each checkpoint serves. A checkpoint designed for accountability, where a person signs off a final report, looks very different from one designed to prevent a wrong payment.
Pattern 1: risk-based approval
Not all actions need the same level of oversight. Classify actions by reversibility and impact, and require approval only where both justify it.
| Action type | Example | Oversight |
|---|---|---|
| Read-only | Search orders, read documents | None; log only |
| Reversible, low impact | Draft an email, add an internal note | None or async review |
| Reversible, higher impact | Update a customer record | Approval for unusual cases only |
| Irreversible or external | Send an email, issue a refund, delete data | Always approve, or approve above a threshold |
| Very high impact | Large payments, legal commitments | Approval plus second reviewer |
Thresholds keep volume manageable: refunds under €50 might go through automatically with logging, while larger ones require approval. Enforce the policy inside the tools, not just in the prompt; see designing tools that agents can actually use.
Pattern 2: confidence-based routing
Route outputs to humans when the system is unsure. Signals include validation failures, disagreement between two model runs, low classifier confidence, rare categories, or inputs that match known difficult patterns. Everything else proceeds automatically, with sampling for quality control.
This pattern works well for high-volume tasks such as document processing or ticket triage, where most cases are routine and a small share needs judgement.
Pattern 3: draft and approve
The agent prepares the complete action, such as an email, a refund or a code change, and a human approves, edits or rejects it. This is the most natural pattern for external communication and is easy for users to understand.
The key is making approval fast and meaningful: the draft must be complete, the relevant context visible, and editing possible without starting over.
Pattern 4: batched review
Instead of interrupting someone for each item, collect items needing approval and present them together: a morning queue of proposed actions, sorted by risk. Batching reduces context switching and lets reviewers compare similar cases. It suits work that is not time-critical.
Pattern 5: ask, don't guess
Many agent failures come from guessing missing information. Give the agent an explicit "ask the user" tool and clear triggers for when to use it: ambiguous requests, missing required details, conflicting instructions. A good question early saves a wrong action later.
Keep questions efficient: ask for everything needed at once, offer choices where possible, and explain briefly why the information is needed.
Pattern 6: escalation with context
When an agent stops, whether because of a budget, an error or a request outside its scope, hand the case to a human with everything needed to continue: what the user asked, what the agent did, what it found and where it stopped. A handoff without context makes the human start from scratch and makes the AI feel like an obstacle. See step budgets and stopping rules for graceful stopping.
Designing review screens people actually read
Approval fatigue is the main way HITL fails. People who approve dozens of correct actions stop reading the next one. Good review design fights this:
- Show the decision, not the transcript. Lead with the proposed action, its key parameters and its consequences: "Refund €420 to card ending 4471 for order A-1029."
- Highlight what is unusual. Flag amounts above normal, new recipients, first-time actions or policy exceptions.
- Show the evidence that supports the action, such as quotes from the conversation or the policy clause, in one click, not ten.
- Make rejection and editing as easy as approval, with a reason field that feeds back into evals.
- Keep the approval rate meaningful. If humans approve 99.9% of a category, consider automating it with sampling. If they reject a lot, fix the agent before adding more checkpoints.
Who should be in the loop
The right reviewer depends on the purpose of the checkpoint. Users themselves are the best reviewers for actions taken on their behalf, such as sending their email or booking their meeting, because they know their intent. Operations staff suit policy decisions such as refunds and exceptions. Domain experts are needed for professional judgement in legal, medical or financial content. Mixing these up causes problems: asking end users to approve policy exceptions invites abuse, and asking operations staff to approve a user's personal email wastes their time.
Make sure reviewers have the authority and the information to say no. A reviewer who cannot see the evidence, or who feels pressure to keep the queue moving, provides the appearance of oversight without the substance.
Measuring whether HITL works
Track:
- approval rate and edit rate per action type;
- time to decision, to spot fatigue and bottlenecks;
- errors that passed review, the most important metric, found through audits and customer reports;
- automation candidates: action types with consistently near-perfect approval.
Periodically insert known-bad test items into review queues to check that reviewers catch them. It is an uncomfortable but effective way to measure real oversight rather than assumed oversight.
HITL and agent design
Human checkpoints are easiest to add when the architecture supports them: actions separated into prepare and commit steps, state persisted so a run can pause and resume, and clear records of what was approved. These are also the properties that make agents easier to debug; see AI agent failure modes. If your agent needs human approval at almost every step, it may be better built as a workflow; see do you need an agent, or just a workflow?.
Frequently asked questions
What is human-in-the-loop AI?
Human-in-the-loop AI is a design in which people review, approve, correct or supply information to an AI system at defined points, typically before high-impact actions or when the system is uncertain.
Which agent actions should require human approval?
Actions that are irreversible, external or high-impact, such as payments, deletions, legal commitments and messages to customers, especially above defined thresholds. Read-only and easily reversible actions usually do not need approval.
How do I avoid approval fatigue?
Approve by risk rather than by default, batch non-urgent reviews, show the decision and its unusual features clearly, and automate categories that are almost always approved, with sampling for quality control.
How do I know if human review is actually catching errors?
Track errors that passed review, audit samples of approved actions, and periodically insert known-bad test items into review queues to measure whether reviewers catch them.
