Few-shot prompting means including a handful of worked examples (input and desired output) in your prompt, so the model can infer the task from demonstrations. It is one of the oldest and most effective prompting techniques, and it is the reason the GPT-3 paper was titled "Language Models are Few-Shot Learners."
Examples are also one of the most common sources of subtle prompt bugs. Models copy examples very faithfully: their length, phrasing, structure and even their specific content. A well-chosen example can fix a behaviour that pages of instructions could not; a careless one can introduce a bias you will spend weeks tracking down. This guide explains what models actually learn from examples, the known pitfalls, and how to choose, order and test them.
What models learn from examples
Research on in-context learning has produced some surprising findings. Min et al. (2022), "Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?", found that for classification-style tasks, replacing the correct labels in examples with random labels hurt performance much less than expected. What mattered more was that examples showed the format, the label space (which answers are possible) and the distribution of inputs.
In practice, this means examples teach models mostly about:
- the shape of the output: length, structure, tone, formatting;
- the set of possible answers: which categories, which fields, which styles;
- the kind of input the task involves.
They are less effective at teaching subtle reasoning or rules than you might hope, and very effective at teaching surface patterns, including ones you did not intend.
How examples backfire
Copying content
If your example email reply mentions a "10% discount," expect to see "10% discount" in outputs where it does not belong. Models treat example content as a hint about what good answers contain.
Copying length and structure
A single long example produces long outputs; three-bullet examples produce three bullets, even when one or five would be right.
Label bias
Zhao et al. (2021), "Calibrate Before Use," documented several biases in few-shot classification:
- majority label bias: the model favours labels that appear more often in the examples;
- recency bias: it favours the label of the last example;
- common token bias: it favours answers that are common words in general.
Order sensitivity
Lu et al. (2022), "Fantastically Ordered Prompts and Where to Find Them," showed that simply reordering the same examples could swing accuracy substantially for some models and tasks.
Never using null or "other"
If every example fills every field or assigns a specific category, the model learns that "unknown" is not an acceptable answer and starts inventing values. This is one of the most common causes of hallucinated fields in extraction; see structured output without JSON mode.
How to choose good examples
- Cover the hard cases, not the obvious ones. Instructions handle the easy majority. Use examples for ambiguous inputs, edge cases and refusals.
- Balance labels. For classification, include each important label roughly equally, and do not end with the most common one.
- Include a "none" example. Show at least one case where the correct output is null, "other" or "I don't know."
- Vary surface features. Different lengths, phrasings, topics and formats keep the model from latching onto one pattern.
- Use realistic inputs. Synthetic examples that are cleaner than real data teach the wrong input distribution.
- Keep outputs exactly as you want them. Every detail of example outputs, including punctuation and length, is an implicit instruction.
- Use neutral content. Avoid distinctive names, numbers and offers that could leak into outputs.
How many examples?
There is no universal number. Some practical guidance:
| Situation | Suggested approach |
|---|---|
| Strong model, simple format | Zero-shot with a clear template; add examples only if needed |
| Strong model, tricky edge cases | 1–3 examples focused on the edge cases |
| Small or open model | 3–8 examples; small models benefit more from demonstrations |
| Classification with many labels | At least one per important label, or describe labels and show a few hard cases |
| Strict output format | One complete example, plus schema validation in code |
More examples cost tokens on every request, and returns diminish quickly. If you need dozens of examples to get acceptable behaviour, consider fine-tuning, or retrieving a few relevant examples dynamically for each input.
Dynamic examples
Instead of fixed examples, retrieve the most similar labelled examples from a pool for each new input, using embeddings or keywords. This tends to improve accuracy because examples are relevant to the case at hand, and it avoids biases from a single fixed set. Make sure the pool is balanced and clean, and that retrieval cannot return the exact test input during evaluation.
Formatting examples clearly
Separate examples from instructions and from the real input with consistent delimiters, so the model does not confuse them:
<examples>
<example>
<input>My card was charged twice for the March invoice.</input>
<output>{"category": "billing", "urgent": false}</output>
</example>
<example>
<input>Can't log in since this morning and payroll is due at noon!</input>
<output>{"category": "account", "urgent": true}</output>
</example>
<example>
<input>Do you sponsor community events?</input>
<output>{"category": "other", "urgent": false}</output>
</example>
</examples>
<input>{user_message}</input>Label the examples clearly as illustrations. If the content of an example must not be reused, say so: "These examples show the format only; do not reuse their details."
Testing examples properly
Treat examples as part of the prompt and test them the same way:
- Run your eval set with and without examples, and with two or three different example sets.
- Check the label distribution of outputs against the true distribution; a skew towards example labels signals bias.
- Search outputs for example content (specific names, numbers, phrases) to detect copying.
- Shuffle example order and re-run; large swings mean the prompt is fragile.
- Read failures on inputs that resemble examples and on inputs that do not.
For where examples fit in a full prompt, see anatomy of a production system prompt. And remember that examples showing what *not* to do can backfire in the same way as negative instructions; see why "don't do X" instructions fail.
When to stop adding examples
Examples are a lever with limits. If you find yourself adding a new example for every new failure, the prompt is turning into a training set. That is a sign to consider fine-tuning on a proper dataset, using a larger model, or routing difficult inputs elsewhere. For narrow, high-volume tasks, a small fine-tuned model often beats a large prompt full of examples; see small models, big jobs.
Frequently asked questions
What is few-shot prompting?
Few-shot prompting includes a small number of example inputs and desired outputs in the prompt, so the model can infer the task and the expected format from demonstrations rather than from instructions alone.
How many few-shot examples should I use?
Often between one and five for strong models, focused on difficult cases, and somewhat more for smaller models. Test with your eval set; returns diminish quickly and each example adds cost to every request.
Why does my model copy details from the examples?
Models treat example outputs as strong signals of what good answers look like, including their content. Use neutral content, vary examples, and state that examples illustrate format only.
Does the order of few-shot examples matter?
Yes. Research has shown that example order can change accuracy noticeably, and models tend to favour the label of the last example. Balance labels and test several orders.
