Methodology

Bradley–Terry calculation

P(A beats B) = 1 / (1 + 10^((Elo_B − Elo_A) / 400))

Internal testers.
Battles where either contestant failed.
Battles where either contestant has $0 recorded account spending, including sponsored provider endpoints.
Incomplete records.
Battles affected by the historical subagent bug.
With Exclude early voting: early picks and battles stopped before both contestants finished.

Developer-cluster bootstrap; B = 1,000 replicates.
Each replicate draws D developers uniformly with replacement from D observed developers, retaining all their battles and multiplicity.
Refit the maximum a posteriori Bradley–Terry estimate for each replicate.
Tied rank positions split their weight equally.
Ridgeline smoothing: Gaussian kernel density estimates, with bandwidth h = max(2, 1.06sB^−1/5) Elo; s is the replicate standard deviation.
Ridgeline axis: from the lowest 0.5th to the highest 99.5th percentile of the ranked models' curves, rounded out to 25 Elo.

OpenRouter routes each model through a model-specific provider and endpoint allowlist.
Routing prioritizes throughput, requires support for the requested parameters, and applies per-model price caps.

Rank likelihood: share of bootstrap samples at each rank.
Ridgelines: smoothed Elo distributions from bootstrap samples.
Raw win rate: excludes ties from wins.
Tokens per second: median per-run token rate.
Cost / time: median or arithmetic mean per task, using recorded values from eligible battles.
Pareto frontier: ranked models with no other ranked model having equal or higher Elo and equal or lower selected cost or time, with at least one strict improvement.
Pareto bars: 95% Elo confidence intervals.
Thin / thick bars: thin bar is the 95% confidence interval; thick bar is the 50% confidence interval.