A model outputs a probability, not a winner
A sports prediction model turns data — team strength, recent form, starting pitchers or quarterbacks, home field, rest, injuries — into a probability for each outcome. It does not say who will win. It says how often a team in this situation wins.
That is why "the model was wrong" rarely makes sense about a single game. A 70% favorite loses three times in ten. The right question is whether the model's probabilities hold up across many games.
Calibration: do the numbers mean what they say?
A calibrated model's 60% calls win about 60% of the time, its 70% calls about 70% of the time, and so on. Calibration is checked by grouping forecasts by their probability and comparing each group with what actually happened.
A model can call plenty of winners and still be poorly calibrated — for example, by claiming 80% on games that only win 65% of the time. Those overconfident probabilities would mislead anyone using them to judge a price.
The Brier score
The Brier score measures accuracy and calibration together. For each forecast, take the probability given to the outcome, subtract 1 if it happened or 0 if it didn't, and square the result. Average those numbers across all forecasts. Lower is better.
A 70% forecast that wins scores (0.7 − 1)² = 0.09. If it loses it scores (0.7 − 0)² = 0.49. A forecaster who always says 50% scores 0.25 on every two-sided game, so a useful model needs to average below that — and, more to the point, below the simple baseline it is meant to improve on.
Held-out evaluation
A model can look brilliant on the games it was built from and still fail on new ones. The fix is a held-out evaluation: grade the model on a season of games it never saw during development, and require it to beat a baseline before it is released.
SharpDeck's NFL pregame model was released that way: evaluated on 543 regular-season games it had never seen, it beat the model it replaced on both Brier score and accuracy. Models that pass that gate carry a "Validated model" label.
The public record is the final check
Held-out evaluations happen before release. After release, the only honest evidence is a forecast recorded before the game and graded after it, with the losses left in. That is why every SharpDeck forecast is published at fixed checkpoints — 24 hours and 60 minutes before the start, and live where available — and graded in a public record anyone can inspect.
When you evaluate any prediction service, ask three questions: were the forecasts published before the games, are the losses still visible, and is the sample large enough to mean something?
SharpDeck field notes. Product explanations and worked examples, not betting recommendations.