Brier score
How it works
1) For each example take the vector of predicted probabilities and the one-hot vector of the true class (for binary: a single p and a 0/1 outcome). 2) Compute the squared difference between the forecast and the outcome (for multiclass: the sum of squares over classes). 3) Average over all examples — the result is the Brier score. 4) Interpret: lower is better; 0 is a perfect forecast. 5) Optionally apply Murphy's decomposition to separate the contributions of calibration (reliability), resolution, and irreducible uncertainty.
Problem solved
It solves the problem of evaluating forecasts that are probabilities rather than hard decisions: it lets models be compared on how well their probabilities match reality and — as a proper scoring rule — it rewards honest reporting of uncertainty rather than overconfidence.
Components
The predicted class probabilities (or a single p in the binary case).
The actual outcome encoded as 0/1 (or a one-hot vector of the true class).
The squared difference between forecast and outcome, summed over classes in the multiclass version.
The decomposition of the Brier score into reliability, resolution, and uncertainty, giving insight into error sources.
Official
Evolution
Glenn Brier defines the measure for verifying probabilistic weather forecasts.
Allan Murphy decomposes the Brier score into reliability, resolution, and uncertainty.
Gneiting and Raftery place the Brier score within the general theory of strictly proper scoring rules.
Hyperparameters (configurable axes)
Binary version (single p) vs multiclass (sum of squares over classes); determines the value range and interpretation.
Computational complexity
Time complexity: O(n·K). Space complexity: O(1) przy agregacji strumieniowej; O(K) na przykład.
Parallelism
Ideal for map-reduce computation and vectorization.
Hardware requirements
The Brier score is a simple vector operation (squared difference and mean); it runs efficiently on any hardware.