Robots Atlas>ROBOTS ATLAS
Evaluation

Brier score

1950ActivePublished: 17 November 2026Updated: 17 November 2026Published
Key innovation
As a strictly proper scoring rule, it rewards probabilistic forecasts that are both accurate and honest — the minimum is achieved only by reporting the true probability.
Category
Evaluation
Abstraction level
Pojęcie
Operation level
Evaluation (runtime)Inference
Use cases
Assessing the quality of probabilistic forecasts (weather, risk, classification)Comparing models on calibration and resolution at onceHonestly scoring forecasters and prediction marketsMonitoring probability reliability alongside ECE and log loss

How it works

1) For each example take the vector of predicted probabilities and the one-hot vector of the true class (for binary: a single p and a 0/1 outcome). 2) Compute the squared difference between the forecast and the outcome (for multiclass: the sum of squares over classes). 3) Average over all examples — the result is the Brier score. 4) Interpret: lower is better; 0 is a perfect forecast. 5) Optionally apply Murphy's decomposition to separate the contributions of calibration (reliability), resolution, and irreducible uncertainty.

Problem solved

It solves the problem of evaluating forecasts that are probabilities rather than hard decisions: it lets models be compared on how well their probabilities match reality and — as a proper scoring rule — it rewards honest reporting of uncertainty rather than overconfidence.

Components

Forecast probability vectorThe input being scored.

The predicted class probabilities (or a single p in the binary case).

One-hot outcomeThe reference for the squared error.

The actual outcome encoded as 0/1 (or a one-hot vector of the true class).

Squared errorThe core of the measure.

The squared difference between forecast and outcome, summed over classes in the multiclass version.

Murphy decompositionDiagnostics: where the metric's value comes from.

The decomposition of the Brier score into reliability, resolution, and uncertainty, giving insight into error sources.

Official

Evolution

Original paper · 1950 · Monthly Weather Review · Glenn W. Brier
Verification of Forecasts Expressed in Terms of Probability
Glenn W. Brier
1950
Introduction of the Brier score
Inflection point

Glenn Brier defines the measure for verifying probabilistic weather forecasts.

1973
Murphy's decomposition
Inflection point

Allan Murphy decomposes the Brier score into reliability, resolution, and uncertainty.

2005
Formalization of proper scoring rules

Gneiting and Raftery place the Brier score within the general theory of strictly proper scoring rules.

Hyperparameters (configurable axes)

Formulation (binary/multiclass)Medium

Binary version (single p) vs multiclass (sum of squares over classes); determines the value range and interpretation.

BinarnaRange [0,1].
WieloklasowaRange [0,2] under the one-hot sum-over-classes convention.

Computational complexity

Time complexity: O(n·K). Space complexity: O(1) przy agregacji strumieniowej; O(K) na przykład.

Parallelism

Parallelism level
Fully parallel

Ideal for map-reduce computation and vectorization.

Scope
InferenceAcross devices
Constraints
!Squared errors are independent per example; only a final reduction (sum and mean) is needed.

Hardware requirements

Primary

The Brier score is a simple vector operation (squared difference and mean); it runs efficiently on any hardware.