A reward model is trained on the outcome signal; via a log-likelihood parameterization, per-step rewards emerge implicitly.
Training explicit PRMs requires expensive per-step labels for each reasoning step.
