Skip to content
All tags

#arena

5 posts

Benchmark Watch | Arena Agent Leaderboard: After Seven Days Stuck on the Same Batch, Gemini 4 Carbon Debuts at #2, Pushing Sonnet 5.5 to Third

New Arena Agent leaderboard batch (official publish date 2026-10-28, ending seven straight days stuck on the 10-23 batch): first place Claude Opus 5.5 (Max) holds at 20.94%; Google's Gemini 4 Carbon (High), announced on 10-28 and currently limited to cyber-defender testers, debuts straight into second place at 16.04%, pushing previously-second Claude Sonnet 5.5 (Max) to third (13.99%→13.84%); GPT-6 Astra (Max) falls hardest, from third to fifth (13.96%→12.96%), overtaken by Claude Fable 5.1 (Max), whose rank stays fourth but whose score dropped less (13.94%→13.35%). Carbon's 95% CI lower bound (15.04%) still sits above Sonnet's point estimate (13.84%) — one of the few rank gaps in this batch that actually holds up statistically.

Benchmark Shift | Arena Agent Leaderboard: After Three Days Stuck on the Same Batch, Claude Sonnet 5.5 Jumps From Fifth to Second, Pushing Fable 5.1 Out of the Top Three

New Arena Agent leaderboard batch (official publish date 2026-10-23, ending three straight days stuck on the 10-20 batch): first place Claude Opus 5.5 (Max) holds at 20.77%; Claude Sonnet 5.5 (Max) jumps from fifth at 12.67% to second at 13.99% (+1.32pp), pushing previously-third Claude Fable 5.1 (Max) out of the top three (14.17%→13.94%, down to fourth); GPT-6 Astra (Max) drops from second to third (14.70%→13.96%). Second through fifth are now only 0.35 percentage points apart, and all their 95% confidence intervals overlap — but Sonnet 5.5 matching Fable 5.1 at one-fifth the API price is the detail worth keeping.

Benchmark Shift | Arena Agent Leaderboard: GPT-6 Astra Breaks Into the Top Three After Four Batches Stuck Below It, GPT-6.1 Sol Falls From a Three-Batch Hold on Second to Fourth

Arena Agent leaderboard published 10-16: Claude Opus 5.5 (Max) stays first at 18.88%; GPT-6 Astra (Max) breaks into second at 14.13% after three straight batches stuck at 4th or 5th; GPT-6.1 Sol (Max) ends a three-batch hold on second, dropping to fourth (14.78%→13.60%). The top six's 95% confidence intervals still overlap heavily, but Astra's score rose across four straight batches and Sol's long stable run just ended — a multi-batch positional shift like this is worth more than a single batch's rank jump.

Benchmark Shift | Arena Agent Leaderboard: Claude Fable 5.1 Back in Third, GPT-6 Sol Drops From Third to Seventh in a Day

Arena Agent leaderboard published 10-13: Claude Opus 5.5 (Max) stays first at 17.01%; GPT-6.1 Sol (Max) second at 15.05%; Claude Fable 5.1 (Max) climbs from sixth back to third (11.74%→14.32%); GPT-6 Sol (Max) falls from third to seventh (12.60%→10.51%). Three releases in one week, three changes in third place, and overlapping 95% confidence intervals across the top six — the ranks themselves shouldn't drive model selection.

Funding Alert: Arena Raises $200M Series B

Arena closed a $200M Series B at a $3.1B valuation, nearly double where it stood ten months ago, co-led by Lightspeed Venture Partners and Khosla Ventures. The signal: once models can recognize they're being tested, static benchmarks stop telling the truth, and a third-party, real-time, human-judged evaluation layer becomes a business of its own.