You can’t game an honest score: a visual intuition for proper scoring rules
Why honesty is mathematically the best strategy under a Brier score — in one picture. Hedging to “50%” doesn’t protect you; it just locks in a guaranteed middling penalty.
There’s a reason AEQUARA scores everything with a Brier score and not, say, plain accuracy. A Brier score is a proper scoring rule, which is a precise mathematical way of saying: your best move is to report what you actually believe. You can’t hedge your way to a good number. Here’s the intuition in one picture.
Reading the picture
The green curve is your penalty if the event happens: the more confident you were, the smaller it is. The bronze curve is your penalty if it doesn’t: the more confident you were, the larger it is. The curves cross at 50%, and that crossing point is the trap.
Why hedging doesn’t save you
Say “50%” to play it safe and you lock in a penalty of 0.25 no matter what reality does. You’ve bought certainty about your score by giving up all hope of a good one. To get below 0.25 you have to lean — say 70%, or 90% — and the only way leaning pays off on average is if your confidence tracks reality. Overclaim and the wrong-side curve punishes you hard; underclaim and you leave score on the table. The optimum is to report your true belief.
That’s the whole reason we use it
A scoring rule you can game would reward confident nonsense. A proper one rewards honesty and nothing else, which is exactly the property you want at the centre of a company built on calibrated truth. It’s why the AI Trust Index scores models this way, and why the free Calibration Scorecard can tell you something real about your own judgment in two minutes — there’s no way to fake a good result, including for us.