Inference

Prefill

2024ActivePublished: 10 September 2026Updated: 10 September 2026Published

Key innovation

Lets you steer a model's output by writing the opening of the assistant turn for it to continue from — with no fine-tuning or weight changes.

How it works

1. In the messages array, the final entry uses the "assistant" role and contains partial text (the prefix). 2. The model treats this text as the beginning of its own reply and generates the continuation directly from it. 3. The prefix conditions the probability distribution of subsequent tokens, so the first characters (e.g. {, [Name], a heading) impose format and tone. 4. Documentation example: for a multiple-choice question you prefill "The best answer is (" and the model continues, e.g. "B)". 5. Prefill applies only to the last assistant turn; earlier assistant turns in the conversation are unaffected.

Problem solved

Language models often add unnecessary preambles, drift from the required output format, break the assigned role, or refuse unnecessarily — making deterministic, machine-parseable output hard to obtain. Prefill gives direct control over the start of the reply without fine-tuning.

Components

Assistant turn prefixSets the start of the reply and imposes the format and tone of the continuation.

Partial text placed in the final assistant-role message, from which the model continues generating.

Official

Model continuationThe model's actual response, constrained by the shape of the prefix.

Tokens generated by the model immediately after the prefix, conditioned on its content.

Implementation

Reference implementations

Anthropic Messages API — prefill via assistant role

HTTP / JSON · Anthropic

Official

Implementation pitfalls

Removed in Claude 4.6+ modelsCritical

Prefilling the last assistant turn returns a 400 error on Claude 4.6+ and Mythos Preview.

Fix:Instead of prefill, use Structured Outputs, system-prompt instructions, or tool calling.

Over-constraining the responseMedium

Prefill constrains part of the model's response, so a poorly chosen prefix can force an incorrect structure or content.

Fix:Keep the prefix short and neutral; limit it to the minimum needed to impose the format.

Prefill only on the last turnLow

Prefill works only when the prefix is in the final assistant message; earlier turns do not use it.

Fix:Place the steering prefix as the final assistant-role message.

Evolution

2023

Continuation from an assistant turn in the Messages API

Inflection point

The Claude Messages API enables continuing a response from a partial assistant-role message — the technical basis of prefill.

2024

Prefill documented as a prompting technique

Anthropic documents prefill in its prompt-engineering guide as a way to control format, skip preambles, and maintain a role.

2026

Removal of last-turn prefill in Claude 4.6+

Inflection point

Per current Anthropic documentation, last-turn assistant prefill is unsupported on Claude 4.6+ and Mythos Preview (400 error); recommended alternatives are Structured Outputs, system-prompt instructions, and tool calling.

Prompting best practices — Migrating away from prefilled responses (paper)