Evaluation

LLM Judge

2023ActivePublished: 10 September 2026Updated: 10 September 2026Published

Key innovation

Showed that a strong language model can evaluate open-ended responses in line with human preferences (over 80% GPT-4–human agreement on MT-Bench), enabling automated, faster, and cheaper evaluation of generative AI systems where lexical metrics fail and human rating is slow and expensive.

How it works

The LLM judge is given an evaluation prompt made of: (1) a role instruction and evaluation criteria (a rubric), (2) context — the question or task, (3) the response to be judged (or two responses in pairwise mode), and optionally (4) a reference answer. The model first produces a rationale (step-by-step, chain-of-thought reasoning) and only then emits a verdict — a number on a small integer scale (e.g. 1–5), a winner label (A/B/tie), or a score following an additive rubric. The output is usually forced into a structure (e.g. JSON) to be parseable. To reduce biases, practitioners swap the response positions and average (position bias), normalize for length (verbosity bias), use a judge different from the evaluated model (self-enhancement bias), and grade against a reference answer for precision-demanding tasks. Multiple judges or multiple runs can be aggregated (a jury/panel of judges, voting).

Problem solved

Assessing the quality of open-ended generative outputs is hard: lexical metrics (BLEU, ROUGE) correlate poorly with human preferences, while reliable human rating is slow, costly, and hard to scale. LLM-as-a-Judge provides automated, fast, and cheap evaluation that, with a strong judge model, achieves high agreement with human ratings and scales to thousands of comparisons.

Components

Judge model (LLM)Judgment generator

A strong language model (e.g. GPT-4, Claude, Prometheus) that produces the judgment. Its reasoning ability determines the reliability of the evaluation.

Proprietary judgeStrong API models (GPT-4, Claude) — high quality but costly and opaque.

Open-source judgeModels trained for evaluation (Prometheus, JudgeLM) — cheap, reproducible, tunable.

Official

Evaluation prompt and rubricDefinition of evaluation criteria

The instruction defining evaluation criteria, scale, and output format. May include an additive rubric and few-shot examples.

INA natural-language prompt with criteria and the answer format.

Scoring protocolHow the judgment is expressed

The choice of mode: pointwise, pairwise comparison, or reference-guided grading.

PointwiseA single response rated on a numeric scale.

PairwiseComparing two responses and choosing the better one.

Reference-guidedGrading against a reference answer, useful for tasks with a single correct answer.

Official

Rationale (chain-of-thought)Reasoning preceding the score

Reasoning steps generated before the verdict. They improve the quality and transparency of the judgment (the G-Eval paradigm: CoT + form-filling).

Official

Aggregation and bias mitigationNoise and bias reduction

Techniques that increase reliability: position swapping and averaging, a panel of judges, voting, length normalization.

Official

Implementation

Reference implementations

FastChat — LLM Judge (MT-Bench)

Python · LMSYS Org

Official

Prometheus-Eval

Python · Prometheus-Eval

RAGAS — LLM-based RAG evaluation

Python · Exploding Gradients

Hugging Face Cookbook — LLM Judge

Python · Hugging Face

Implementation pitfalls

Position biasHigh

In pairwise mode the judge often favors the response in the first (or second) position regardless of quality.

Fix:Judge both orders (A,B) and (B,A) and average the verdicts; mark ties when they disagree.

Verbosity biasMedium

The judge tends to favor longer, more elaborate responses even when they are not better.

Fix:Explicit conciseness criteria in the rubric, length control/normalization, reference-guided grading.

Self-enhancement biasHigh

The judge model may favor responses generated by itself or by models from the same family.

Fix:Use a judge different from the evaluated model; a panel of judges from different providers.

Limited reasoning abilityMedium

The judge may misgrade tasks requiring precise reasoning (math, logic), rewarding incorrect answers.

Fix:Reference-guided grading, forcing CoT, using tools/verifiers.

Scale and calibration instabilityMedium

Absolute ratings on a wide scale (0–10) are less stable and correlate more weakly with humans.

Fix:A small integer scale (1–5), an additive rubric, few-shot anchors, pairwise comparison instead of absolute scores.

Non-determinism and prompt sensitivityMedium

Small prompt changes or temperature > 0 change verdicts and hurt reproducibility.

Fix:Temperature 0, a fixed prompt template, forced structured output (JSON), prompt versioning.

Evolution

Original paper · 2023 · NeurIPS 2023 (Datasets and Benchmarks Track) · Lianmin Zheng

Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, Ion Stoica

2022

RLAIF / Constitutional AI — AI feedback

Inflection point

Anthropic shows an AI model can generate preference labels and feedback for alignment (RLAIF), a direct precursor to using an LLM as an evaluator.

CAI (concept)Constitutional AI: Harmlessness from AI Feedback (paper)

2023

G-Eval — CoT + form-filling for NLG evaluation

"G-Eval" uses GPT-4 with a reasoning chain and a scoring form, achieving better human correlation than lexical metrics on NLG tasks.

G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment (paper)

2023

MT-Bench and Chatbot Arena — formalization and bias analysis

Inflection point

Zheng et al. introduce MT-Bench and Chatbot Arena, showing over 80% GPT-4–human agreement and systematizing position, verbosity, and self-enhancement biases.

Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (paper)

2024

Prometheus — open, fine-grained evaluator

Prometheus (ICLR 2024) is an open 13B model trained on the Feedback Collection that grades with custom rubrics and matches GPT-4 correlation, decoupling evaluation from proprietary models.

Prometheus: Inducing Fine-grained Evaluation Capability in Language Models (paper)

2024

Widespread adoption in benchmarks and RLAIF

LLM-as-a-Judge becomes standard in AlpacaEval, Arena-Hard, RAG evaluation (RAGAS), and as a source of preference data in RLAIF and DPO.

Hyperparameters (configurable axes)

Judge modelCritical

Which LLM acts as the judge. A stronger model means higher human agreement but higher cost; it is important that it not be the evaluated model (self-enhancement bias).

GPT-4Strong reference judge in MT-Bench

Prometheus 13BOpen, tunable judge

Scoring protocolHigh

Pointwise (numeric scale), pairwise (comparing two responses), or reference-guided. Affects stability and susceptibility to position bias.

pairwiseMore stable than absolute scoring

pointwise 1-5A small integer scale improves correlation

Scoring scaleMedium

The range of the pointwise scale. Small integer scales (1–4 or 1–5) yield higher human correlation than a continuous 0–10.

1-5Recommended small scale

Reason-before-score (CoT)High

Whether the model generates a rationale before the verdict. Clearly improves correlation (the G-Eval paradigm).

Reference answerMedium

Including a reference answer. Improves grading of precision-demanding tasks (math, facts).

Position swap (bias mitigation)High

In pairwise mode: judging both orders (A,B) and (B,A) and averaging to neutralize position bias.

TemperatureMedium

Usually set to 0 for reproducibility and deterministic judgments.

0Deterministic judgment

Computational complexity

Time complexity: O(N · L² · d). Space complexity: O(L · d).

Compute bottleneck

Judge model inference cost

Each judgment requires a large-model call; with strong judges (GPT-4) the cost and latency per comparison are high, and in pairwise mode they scale quadratically.

Depends on

Rozmiar modelu-sędziegoDługość promptu i liczba porównań

Execution paradigm

Primary mode

Dense

An inference/evaluation-level pattern using the full dense forward pass of the judge model; it introduces no routing of its own (though a “panel of judges” variant may route cases to different models).

Activation pattern

All paths active

Routing mechanism

Parallelism

Parallelism level

Partially parallel

Individual judgments are independent (embarrassingly parallel) and can be batched across many devices; only generation within a single judgment stays sequential.

Scope

InferenceAcross devices

Constraints

!Generating the rationale and verdict is sequential within a single judgment.