Robots Atlas>ROBOTS ATLAS
Inference

Few-Shot

2020ActivePublished: 10 September 2026Updated: 10 September 2026Published
Key innovation
Demonstrated that a sufficiently large language model can perform a new task from a handful of examples placed directly in the prompt, with no weight updates, removing the need for a separately fine-tuned model per task.
Category
Inference
Abstraction level
Pattern
Operation level
Inference
Use cases
Text classification (e.g. sentiment analysis, categorization)Information extraction and data structuringMachine translationFormatted or style-constrained text generationQuestion answeringRapid prototyping of NLP tasks without collecting training data

How it works

The prompt is assembled as a sequence: (1) an optional instruction describing the task, (2) k demonstrations, each showing an example input paired with its correct output in a consistent format, and (3) the actual input ending with a cue for the model to continue. The whole sequence is fed to the model as a single context; self-attention lets the model relate the new input to the patterns seen in the demonstrations and generate an answer with an analogous structure. No weight update occurs — the "learning" happens purely in the forward pass, within the context activations. Output quality depends on the number of examples (k), their representativeness, their ordering, and the consistency of the format and label verbalizers.

Problem solved

Adapting a model to a new task traditionally requires collecting a labeled dataset and running fine-tuning, which is costly and slow. Few-shot prompting addresses this by letting a pretrained model be adapted to a new task at inference time using only a handful of in-prompt examples, with no training and no separate copy of weights.

Components

Task instructionTask definition

An optional natural-language description stating what the model should do. It may be omitted when the demonstrations alone define the task sufficiently.

Official

Demonstrations (exemplars)Carrier of the task pattern (in-context learning)

A set of k input–output pairs showing correct task solutions in a consistent format. The core element from which the model infers the pattern.

INk examples, each a text pair input → output.
Query (test input)Actual query

The new input for which the model should generate an answer, formatted identically to the demonstration inputs.

Output cue / formatAnswer format control

A marker or label (verbalizer) signaling where and in what form the model should continue generating the answer.

Official

Implementation

Reference implementations
OpenAI Cookbook — few-shot examples
Python · OpenAI
Official
DSPy — programmatic few-shot optimization
Python · Stanford NLP
LangChain — FewShotPromptTemplate
Python · LangChain
Implementation pitfalls
Sensitivity to example orderingHigh

Reordering demonstrations can significantly change the prediction (majority-label and recency biases).

Fix:Average over multiple permutations, calibrate the output, use a careful fixed ordering.
Example selection qualityHigh

Random examples yield worse results than examples well matched to the query.

Fix:Select examples via retrieval (embedding kNN) most similar to the query.
Context window limit and costMedium

A large number of examples exhausts the context window and raises inference cost and latency.

Fix:Compress/summarize examples, retrieve only the best k, cache the prompt.
Format and verbalizer sensitivityMedium

Inconsistent separator or label formatting degrades the model ability to infer the pattern.

Fix:Keep a uniform template and labels across all demonstrations and the query.

Evolution

Original paper · 2020 · NeurIPS 2020 · Tom B. Brown
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Dario Amodei
2020
GPT-3 and formalization of few-shot
Inflection point

"Language Models are Few-Shot Learners" introduces GPT-3 and distinguishes zero-, one-, and few-shot settings, showing that model scale enables in-context learning.

Language Models are Few-Shot Learners (paper)
2021
Example selection matters

"What Makes Good In-Context Examples for GPT-3?" shows that retrieving examples similar to the query significantly improves results over random selection.

What Makes Good In-Context Examples for GPT-3? (paper)
2021
Order sensitivity and calibration

"Calibrate Before Use" documents majority-label, recency, and ordering biases in few-shot prompts and proposes output calibration.

Calibrate Before Use: Improving Few-Shot Performance of Language Models (paper)
2022
Role of demonstrations

"Rethinking the Role of Demonstrations" shows that the format and label space of demonstrations matter more than the correctness of the labels themselves.

Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? (paper)
2022
Few-shot + Chain-of-Thought
Inflection point

Chain-of-Thought prompting extends few-shot with demonstrations containing reasoning steps, markedly improving multi-step reasoning tasks.

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (paper)

Hyperparameters (configurable axes)

Number of shots (k)High

How many demonstrations are placed in the prompt. Typically 1–32; bounded by the context window. More examples usually help up to a saturation point.

1One-shot
5Typical few-shot setting
32Value used in some GPT-3 experiments
Example selection strategyHigh

How demonstrations are chosen: randomly, manually, or by retrieving the most similar to the query (retrieval-based). Selection strongly affects quality.

randomRandom selection
kNN / retrievalNearest neighbors to the query
Example orderingMedium

The order of demonstrations in the prompt. Models can be sensitive to ordering (recency and majority-label bias), affecting stability.

fixedFixed, hand-picked order
Prompt template / formatHigh

The format of separators, labels, and verbalizers joining inputs and outputs. A consistent format is critical for correct pattern inference.

Computational complexity

Time complexity: O(n² · d). Space complexity: O(n · d).

Compute bottleneck

Context length / context window

The number of examples is bounded by the model context window; a longer prompt increases inference cost, latency, and KV-cache memory usage.

Execution paradigm

Primary mode
Dense

An inference-time technique that uses the full dense forward pass of the underlying language model; it introduces no routing of its own.

Activation pattern
All paths active
Routing mechanism

Parallelism

Parallelism level
Partially parallel

Prompt processing (prefill) is parallel across tokens; answer generation remains sequential (autoregressive).

Scope
InferenceAcross tokens

Hardware requirements

Primary

It is an inference-level prompting technique — it runs on any hardware capable of serving the underlying language model.

Good fit

Longer few-shot prompts benefit from high-throughput, high-memory accelerators (Tensor-Core GPUs) during LLM inference.