system-one-benchModel profile
Every figure for one configuration on a single screen: adjusted global score, accuracy per set and per question, calibration, cost, latency and its role in cascades and manipulation alerts. Everything comes from the same scorer as the scoreboard.
Loading…
Adjusted global score
Margin over the baseline of always answering the majority option, averaged per phase: 0 = trivial majority, 100 = perfect. The OOD phase does not enter the mean because its majority is already 100 %.
Accuracy by set and by question
Point percentage with its 95 % CI and, below, the points per question (level questions award half a point one step away). In brackets, what the trivial majority achieves on that question.
Calibration and latency by set
Brier: mean squared error of the yes/no probabilities (0 perfect; 0.25 = always answering 50 %). ECE: mean distance to the calibration diagonal. 0.45–0.55 counts the near coin-flip yes/no answers at the boundary.
Cost and latency
Cost is only what the API charges per 1,000 complete-phase records (local figures do not include hardware or electricity). Latency is the median per message; in cascades, the sum of both passes.
Cascades and manipulation alerts
The role this configuration plays in second passes and, if a manipulation alert was measured for it (threshold ≥ 0.5), how many manipulated messages it detects and how many honest ones it flags by mistake.