Decomposition prompts for long, messy tasks

SSome tasks are too big for one prompt. "Read these 40 customer interviews and write a product strategy." "Review this 120-page contract against our policy and draft redlines." "Migrate this module to the new API and update the tests." You can paste everything into a long-context model and ask for the result, and sometimes it works. When it fails, it fails quietly: missed sections, invented details, a conclusion that does not follow from the evidence, and no way to tell which part went wrong.

Task decomposition breaks a large task into smaller steps, each with its own prompt, inputs and checkable output. It is the most reliable way to get consistent results on long, messy work. This guide covers the main decomposition patterns, when to use each, and how to design the steps.

Why one giant prompt fails

• Attention is limited. Long inputs are not used uniformly; details in the middle get missed. See lost in the middle.
• Too many instructions at once. Each extra requirement competes with the others, and some are dropped.
• No checkpoints. You only see the final output, so you cannot tell whether the error came from reading, analysing or writing.
• Output limits. Long deliverables hit output token limits and get truncated or compressed.
• No reuse. If one part is wrong, you have to re-run everything.

Decomposition addresses all five by giving each step a narrow job and a visible result.

Pattern 1: prompt chaining

A fixed sequence of steps where each step's output feeds the next. Example for turning interview transcripts into a report:

1. Extract key quotes and pain points from each transcript into a structured list.
2. Cluster the extracted items into themes.
3. Prioritise themes by frequency and severity, with supporting quotes.
4. Write the report from the prioritised themes.

Each step is simple enough to test independently, and intermediate outputs can be validated in code (for example, check that every quote appears verbatim in its transcript) or reviewed by a person.

Pattern 2: map-reduce

Apply the same prompt to many pieces independently (map), then combine the results (reduce). Ideal for large documents and collections:

• Map: for each section of the contract, list obligations, deadlines and deviations from policy.
• Reduce: merge the lists, remove duplicates, and produce a summary of issues ranked by risk.

Map steps run in parallel, so map-reduce is often faster than a single long prompt. It also keeps each call short, which improves accuracy on details.

from concurrent.futures import ThreadPoolExecutor

def map_reduce(sections, map_prompt, reduce_prompt, llm):
    with ThreadPoolExecutor(max_workers=8) as pool:
        partials = list(pool.map(lambda s: llm(map_prompt.format(section=s)), sections))
    notes = "\n\n".join(f"<section_notes id={i}>\n{p}\n</section_notes>" for i, p in enumerate(partials))
    return llm(reduce_prompt.format(notes=notes))

Keep section IDs in the map outputs so the final result can point back to where each finding came from.

Pattern 3: plan, then execute

Ask the model to produce a plan first (a list of steps), review or validate it, then execute steps one at a time. Useful when the right steps depend on the input, for example when debugging or researching.

A human or a simple check can approve the plan before execution, which catches misunderstandings cheaply. This is the same idea that makes AI coding assistants more reliable when tasks are scoped into small, reviewable changes; see AI-assisted coding that survives code review.

Pattern 4: least-to-most

Zhou et al. (2022), "Least-to-Most Prompting Enables Complex Reasoning in Large Language Models," proposed first asking the model to break a problem into simpler sub-problems, then solving them in order, using earlier answers for later sub-problems. The technique helped models generalise to problems harder than the examples they were shown. It is a natural fit for tasks with a dependency structure: compute A, then use A to compute B.

Choosing a pattern

These patterns are workflows: the structure is defined by you, not decided by the model at each step. For many tasks a workflow is more reliable and cheaper than an autonomous agent; see do you need an agent, or just a workflow?.

Designing good steps

• One job per step. If a step's instructions include "and also," consider splitting it.
• Structured hand-offs. Pass data between steps as structured output (lists, JSON), not prose, so the next step and your code can rely on it; see structured output without JSON mode.
• Carry provenance. Keep source references (document, section, quote) through every step, so final claims can be traced back.
• Validate between steps. Cheap code checks at each boundary catch errors before they propagate.
• Use the right model per step. Extraction and classification steps can often use small, cheap models; synthesis steps may need a stronger one.
• Make steps idempotent and cache results, so fixing one step does not require re-running everything.

Evaluating a decomposed pipeline

Evaluate each step separately and the pipeline end to end. Step-level evals tell you where problems come from; end-to-end evals tell you whether the whole thing works. When the final output is wrong, trace back through intermediate outputs to the first incorrect step, the same method used in AI agent failure modes.

A worked example: contract review

Here is how a contract review against an internal policy might be decomposed.

1. Split the contract into clauses using its numbering, keeping clause IDs. This is done in code, not by a model.
2. Classify each clause by topic (liability, payment, termination, data protection) with a small, cheap model. Validate that every clause gets exactly one topic.
3. Compare each clause with the policy rules for its topic, one call per clause, producing a structured verdict: compliant, deviation or unclear, with the relevant policy rule and a quote.
4. Check in code that every quote appears in the clause and every cited rule exists in the policy.
5. Draft redlines only for clauses marked as deviations, one call each, with the policy rule as context.
6. Summarise the deviations into a one-page report for the lawyer, grouped by risk.

Every step produces an artefact a reviewer can inspect, and when the lawyer disagrees with a finding, you can see exactly which step produced it.

When not to decompose

Decomposition adds latency, cost and engineering. For tasks that fit comfortably in one prompt and produce consistent results, a single call is simpler. Decompose when you see missed details, inconsistent quality, truncation, or when you need intermediate outputs for review or auditing.

Frequently asked questions

What is prompt chaining?

Prompt chaining splits a task into a sequence of prompts where the output of one step becomes the input to the next. Each step has a narrow job, which makes results more reliable and easier to test.

What is map-reduce prompting?

It applies the same prompt to many chunks of input independently (map), then combines the partial results in a final prompt (reduce). It is well suited to long documents and large collections.

Is decomposition better than using a long-context model?

For tasks that need every detail of a long input, decomposition is usually more reliable because each call focuses on a small part. Long-context models work well when the input is moderate and the task is simple.

How is task decomposition different from an AI agent?

In decomposition, you define the steps in advance, which makes behaviour predictable. An agent decides its own steps at runtime, which is more flexible but harder to test and control.