The prompt is assembled as a sequence: (1) an optional instruction describing the task, (2) k demonstrations, each showing an example input paired with its correct output in a consistent format, and (3) the actual input ending with a cue for the model to continue. The whole sequence is fed to the model as a single context; self-attention lets the model relate the new input to the patterns seen in the demonstrations and generate an answer with an analogous structure. No weight update occurs — the "learning" happens purely in the forward pass, within the context activations. Output quality depends on the number of examples (k), their representativeness, their ordering, and the consistency of the format and label verbalizers.
Adapting a model to a new task traditionally requires collecting a labeled dataset and running fine-tuning, which is costly and slow. Few-shot prompting addresses this by letting a pretrained model be adapted to a new task at inference time using only a handful of in-prompt examples, with no training and no separate copy of weights.
An optional natural-language description stating what the model should do. It may be omitted when the demonstrations alone define the task sufficiently.
Official
A set of k input–output pairs showing correct task solutions in a consistent format. The core element from which the model infers the pattern.
The new input for which the model should generate an answer, formatted identically to the demonstration inputs.
A marker or label (verbalizer) signaling where and in what form the model should continue generating the answer.
Official
Reordering demonstrations can significantly change the prediction (majority-label and recency biases).
Random examples yield worse results than examples well matched to the query.
A large number of examples exhausts the context window and raises inference cost and latency.
Inconsistent separator or label formatting degrades the model ability to infer the pattern.
"Language Models are Few-Shot Learners" introduces GPT-3 and distinguishes zero-, one-, and few-shot settings, showing that model scale enables in-context learning.
"What Makes Good In-Context Examples for GPT-3?" shows that retrieving examples similar to the query significantly improves results over random selection.
"Calibrate Before Use" documents majority-label, recency, and ordering biases in few-shot prompts and proposes output calibration.
"Rethinking the Role of Demonstrations" shows that the format and label space of demonstrations matter more than the correctness of the labels themselves.
Chain-of-Thought prompting extends few-shot with demonstrations containing reasoning steps, markedly improving multi-step reasoning tasks.
Time complexity: O(n² · d). Space complexity: O(n · d).
The number of examples is bounded by the model context window; a longer prompt increases inference cost, latency, and KV-cache memory usage.
How many demonstrations are placed in the prompt. Typically 1–32; bounded by the context window. More examples usually help up to a saturation point.
How demonstrations are chosen: randomly, manually, or by retrieving the most similar to the query (retrieval-based). Selection strongly affects quality.
The order of demonstrations in the prompt. Models can be sensitive to ordering (recency and majority-label bias), affecting stability.
The format of separators, labels, and verbalizers joining inputs and outputs. A consistent format is critical for correct pattern inference.
An inference-time technique that uses the full dense forward pass of the underlying language model; it introduces no routing of its own.
Prompt processing (prefill) is parallel across tokens; answer generation remains sequential (autoregressive).
It is an inference-level prompting technique — it runs on any hardware capable of serving the underlying language model.
Longer few-shot prompts benefit from high-throughput, high-memory accelerators (Tensor-Core GPUs) during LLM inference.
