Alignment
Process Reward Model (PRM)
2023ActiveUpdated: 22 September 2026Published
Key
innovation
Scoring the reasoning process step by step instead of only the final outcome.
Category
Alignment
Abstraction level
Building block
Operation level
Post-trainingInference
Use cases
Mathematical reasoningReasoning RL/rerankingInference-time search
How it works
A PRM assigns a score to each reasoning step; these scores are used for training, reranking, or inference-time search.
Problem solved
Rewarding only the final outcome is sparse and doesn't indicate which reasoning step was wrong.