Skip to content

Benchmarks · Analysis

MathArena Leaderboard: Gemini 4 Carbon Takes #1

Gemini 4 Carbon now leads MathArena at 85.4%, ahead of Claude Fable 5.2 and GPT-6.1 Sol. It costs about 15 times more per problem than Sol despite identical token prices, and its lead over Fable sits inside the confidence intervals.

Metarix Editorial · Nov 15, 2026 · 9 min read

Share

Documentation-based research

MathArena.ai social card: laurel-wreathed pi logo with the tagline Evaluating LLMs on uncontaminated math questions and SRI Lab, ETH Zurich and INSAIT logos
On this page

Gemini 4 Carbon is now the top model on MathArena, the ETH Zurich and INSAIT benchmark built from fresh research-level math problems. In the November 13, 2026 update it scored 85.4% expected performance, ahead of Claude Fable 5.2 at 82.8% and GPT-6.1 Sol at 80.6%, ending a long run of OpenAI models in first place.

The more useful number for builders is in the next column. Carbon costs $15.70 per problem on average, Fable 5.2 costs $40.20, and GPT-6.1 Sol costs $1.07, even though Carbon and Sol charge exactly the same per token. Here is what the new leaderboard actually shows, why the per-problem costs are so far apart, and how to use it when you pick a model for hard reasoning work.

Quick verdict

Gemini 4 Carbon has the best MathArena score, but its lead over Fable 5.2 sits inside the confidence intervals, and it buys about 5 points over GPT-6.1 Sol for roughly 15 times the cost per problem. For most hard-reasoning workloads, keep GPT-6.1 Sol (or another cheap, strong model) as the default and escalate to Carbon only for the problems the default fails, with wall-clock and budget caps in place. Budget for Carbon's per-token price doubling on December 1.

Scroll the table horizontally on smaller screens.

MathArena top three at a glance
ModelExpected performanceExpected cost per problemSetting and harness used
Gemini 4 Carbon85.4% ±3.0%$15.70 ±$1.33High effort, Antigravity CLI, Google API
Claude Fable 5.282.8% ±5.3%$40.20 ±$2.59Max effort, Claude Code, via OpenRouter (Anthropic only)
GPT-6.1 Sol80.6% ±2.7%$1.07 ±$0.018Max effort, Codex, via OpenRouter (OpenAI only)
Best open model: Kimi K3.1 Pro54.3% ±5.8%$3.70 ±$0.27Max effort, Kimi harness, via OpenRouter
Source: MathArena leaderboard and model pages (Nov 13, 2026 update), accessed Nov 15, 2026. Expected cost is the average cost of one run on one problem.

What changed on the MathArena leaderboard

MathArena added results for Claude Fable 5.2 and Gemini 4 Carbon on November 13, according to the update log on its home page. Carbon went straight to first place overall, Fable 5.2 to second, and GPT-6.1 Sol, which had led, dropped to third. MathArena co-author Jasper Dekoninck called it the first time "in a very long time" that a non-OpenAI model held the top spot.

Jasper Dekoninck of MathArena announcing Gemini 4 Carbon at #1, November 13, 2026.

The rest of the top ten is OpenAI and Anthropic: GPT-6 Astra (max) at 78.0%, GPT-6 Sol (max) at 75.9% and Claude Opus 5.5 (high) at 75.3% follow, per the full model list. Google's own Gemini 4 Argon sits 12th at 62.1%, which shows how much Carbon's longer reasoning changes the result within the same model family. The best open-weight model, Kimi K3.1 Pro, is 15th at 54.3%.

What MathArena measures

MathArena uses new problems so that models can't have seen them during training. The overall score currently leans on research-level sets built from arXiv papers published in the past month. ArXivMath asks a model to rederive a precise result from a recent paper. BrokenArXiv hands it a plausible but false statement, often a conjecture a new paper has just refuted, and asks for a proof; full marks go only to models that say the statement is false. ArXivLean asks for formal proofs in Lean. If you are new to reading boards like this, our guide to reading LLM leaderboards covers the general traps.

The lead is narrower than the ranking suggests

MathArena publishes confidence intervals, and they matter here. Carbon's 85.4% ±3.0 spans about 82.4% to 88.4%. Fable 5.2's 82.8% ±5.3 spans about 77.5% to 88.1%, so the two overlap almost entirely. Carbon against Sol (77.9% to 83.3%) overlaps only slightly. On MathArena's own numbers, first and second place are statistically close to a tie; first and third are a narrow but real gap.

Coverage also differs. Carbon's model page lists nine results, including ArXivLean and three months each of ArXivMath and BrokenArXiv. Fable 5.2's page lists four: ArXivMath and BrokenArXiv for September and October only. MathArena fills missing results with an item-response model, which is why Fable's interval is wider. Its ranking could move as more results come in.

Scroll the table horizontally on smaller screens.

Head-to-head on the same MathArena sets
CompetitionGemini 4 CarbonClaude Fable 5.2GPT-6.1 Sol
ArXivMath 10/202693.62%, $5.2993.62%, $22.2890.43%, $1.21
ArXivMath 09/202696.23%, $6.0496.23%, $24.1391.51%, $0.74
BrokenArXiv 10/202682.72%, $18.0673.46%, $39.7880.25%, $1.30
BrokenArXiv 09/202682.39%, $18.6080.50%, $33.4482.70%, $1.15
ArXivLean 09/2026 (formal proofs)62.26%, $45.11No result listed35.85%, $1.16
Source: MathArena model pages for each model, accessed Nov 15, 2026. Accuracy and average cost per problem per run. Single-month results carry wide intervals (roughly ±5 to ±13 points), so treat small gaps as noise.

Read across the rows and the picture is consistent. On ArXivMath the three models are within a few points. On BrokenArXiv, Sol matches or nearly matches Carbon at a fraction of the cost. Carbon's clearest lead is formal proof writing in Lean, where it scored 62.26% against Sol's 35.85% on the one set both attempted, the only row where the gap is clearly larger than the noise.

Same price per token, 15 times the price per problem

Gemini 4 Carbon and GPT-6.1 Sol have identical list prices today: $2 per million input tokens, $0.10 per million cached input tokens and $10 per million output tokens, per Google's Gemini API pricing and OpenAI's pricing page. Prices as of Nov 15, 2026. Yet on MathArena Carbon costs $15.70 per problem and Sol $1.07, about 15 times more. Fable 5.2, at $8 input and $40 output per million tokens on Anthropic's pricing page, costs $40.20, about 38 times Sol.

Scatter chart of MathArena expected performance against expected cost per problem: Gemini 4 Carbon 85.4% at $15.70, Claude Fable 5.2 82.8% at $40.20, GPT-6.1 Sol 80.6% at $1.07
MathArena expected performance against expected cost per problem, log scale. Graphic: Metarix, from MathArena data (Nov 13, 2026 update), Nov 15, 2026.

With equal rates, the whole gap is token volume. On BrokenArXiv overall, MathArena reports 539,577 output tokens for Carbon against 80,413 for Sol, about 6.7 times more, while the cost gap on that set is about 14 times. Our reading is that the rest comes from input and cached context re-sent as the agent works across many turns in its harness; MathArena doesn't break input tokens out on the model pages. The Decoder, citing Dekoninck, reports that Carbon's most common failure is running out of time and that it almost never gives up, in one case launching a script that ran for more than eleven hours without checking on it.

What the price change on December 1 does

Google's pricing page lists Carbon's current rates as valid through November 30, 2026. From December 1 they double to $4 input, $0.20 cached input and $20 output per million tokens. Because every rate doubles, the same MathArena runs would cost about $31.40 per problem, roughly 29 times GPT-6.1 Sol at its current price. We covered the schedule in our Gemini 4 Carbon pricing analysis.

Scroll the table horizontally on smaller screens.

What 1,000 hard problems would cost at MathArena's averages
ModelPer problemPer 1,000 problemsExpected performance
GPT-6.1 Sol (max)$1.07$1,07080.6%
Gemini 4 Carbon (high), through Nov 30$15.70$15,70085.4%
Gemini 4 Carbon (high), from Dec 1about $31.40about $31,40085.4%
Claude Fable 5.2 (max)$40.20$40,20082.8%
Metarix calculation from MathArena expected cost and Google's published December 1 rates, Nov 15, 2026. One run per problem; real workloads differ in prompt size, retries and tool use, so treat this as an order of magnitude, not a quote.

Reasoning effort is a cost dial, not a free upgrade

MathArena's list includes some models at more than one effort setting, and the results are a warning against defaulting to max. Claude Fable 5.1 at high effort scored 73.6% for $21.67 per problem; at max it scored 72.3% for $74.72, more than three times the cost for no gain. GPT-6 Astra at low effort scored 73.3% for $1.07, against 78.0% for $4.03 at max.

The top three were not run at the same setting either: Carbon at high, Fable 5.2 and Sol at max. MathArena's methodology post notes it moved Fable 5.1 from max to high because it regularly spent more than 512,000 output tokens without a single tool call. So these are best-recommended configurations, not like-for-like effort comparisons. If you deploy any of these models, test at least two effort levels on your own tasks before picking one.

How MathArena runs the models, and why agent builders should care

Four-step diagram of how MathArena scores a model: new arXiv problems, sandboxed run in the model native harness, grading, and expected performance aggregation
How MathArena scores a model on its research-level sets. Graphic: Metarix, from MathArena's methodology post and model pages, Nov 15, 2026.

Since September 2026, MathArena runs each model in the coding harness it ships with: the Antigravity CLI for Gemini, Codex for OpenAI models and Claude Code for Claude. Each attempt runs in a Docker container with Python and SageMath but no internet, under a 12-hour time limit and a $100 model-cost budget, four runs per problem. The exact harness versions and settings are published on each model page, and the evaluation code is open source.

That makes MathArena closer to an agent benchmark than an old-style question-and-answer test. The models write and run scripts, check intermediate results and decide when to stop. Three lessons carry over directly to coding and research agents:

  • Cap wall-clock time and spend per task. MathArena needed both limits after runs got out of hand. Carbon's main failure mode is timing out, so a hard timeout is a correctness feature, not just a cost control.
  • Reward "this is wrong" answers. BrokenArXiv gives only partial credit to a model that silently repairs a false statement, and none to one that "proves" it. That is the same failure as a coding agent that edits a test until it passes. If your evals only check that a task completed, add cases where the right answer is to refuse or flag the request.
  • Measure cost per solved task, not per token. Two models with identical rate cards ended up 15 times apart. Our cost per successful task guide shows how to track this in production.

A routing pattern that uses the leaderboard

The numbers point to an escalation setup: try the cheap, strong model first, and pass only failures or low-confidence answers to the expensive one. MathArena's own research found that routing compute by problem difficulty cut cost by 29% at equal accuracy for Lean theorem provers on PutnamBench, per its dynamic compute allocation post. The sketch below is illustrative; we did not run it for this article, and you need your own verifier, such as tests, a checker script or a judge model.

# Illustrative escalation router (pseudo-Python)
ATTEMPTS = [
{"model": "gpt-6.1-sol", "effort": "high", "timeout_s": 1800, "max_usd": 3},
{"model": "gemini-4-carbon", "effort": "high", "timeout_s": 10800, "max_usd": 40},
]
def solve(task):
for cfg in ATTEMPTS:
result = run_agent(task, **cfg) # your harness: Codex, Antigravity CLI, etc.
if result.ok and verify(task, result): # tests, checker or judge
return result
return flag_for_human(task)

At MathArena's averages, if the first model solves most problems, the expensive model only runs on the remainder, so your blended cost stays close to the cheap model's while the hardest cases get Carbon-level effort. Set the escalation budget with the December 1 price in mind.

Which model for which job

Scroll the table horizontally on smaller screens.

Picking a model for hard reasoning after the MathArena update
If you needStart withWhy
Hard math or quantitative reasoning at volumeGPT-6.1 Sol80.6% for $1.07 per problem; within a few points of the leaders on most sets
The highest score on the hardest problems, cost secondaryGemini 4 Carbon#1 overall and a clear lead on formal Lean proofs; price doubles Dec 1
A Claude-based stack for long agentic workClaude Fable 5.2, tested against Opus 5.5Close to Carbon on score but the most expensive per problem; Opus 5.5 scored 75.3% for $3.92
Open weights or self-hostingKimi K3.1 ProBest open model at 54.3%, far below the closed leaders on this benchmark
Coding agents specificallyCheck coding benchmarks tooMathArena is math; see Terminal-Bench and SWE-bench style results before choosing
Metarix assessment based on MathArena results (Nov 13, 2026 update) and official pricing pages, Nov 15, 2026.

Verdict

Gemini 4 Carbon deserves its first place: it is the strongest model on MathArena's fresh research problems and the clear leader on formal proofs. But the leaderboard's cost column is the real story. GPT-6.1 Sol delivers about 94% of Carbon's score for about 7% of the cost per problem at today's prices, and Claude Fable 5.2 costs the most without a statistically clear edge over Carbon (see our Fable 5.2 pricing analysis). Use Sol or a similar model as the default, escalate the failures to Carbon under strict time and spend caps, and rerun the math once Carbon's December 1 price takes effect.

Sources

Tools covered in this article

Frequently asked questions

What is MathArena?
MathArena is a benchmark platform from the SRI Lab at ETH Zurich and INSAIT that tests language models on newly released math problems, including research-level questions drawn from recent arXiv papers, so models can't have memorized them. It reports each model's expected performance with confidence intervals and its average cost per problem, and its evaluation code is open source.
Which model is #1 on MathArena?
As of the November 13, 2026 update, Gemini 4 Carbon (high effort) is first with 85.4% expected performance, followed by Claude Fable 5.2 (max) at 82.8% and GPT-6.1 Sol (max) at 80.6%. The confidence intervals of the top two overlap heavily, so their order is not statistically settled.
Why does Gemini 4 Carbon cost more than GPT-6.1 Sol on MathArena if the token prices are the same?
Both models list $2 per million input tokens and $10 per million output tokens as of November 15, 2026, so the difference is how many tokens each uses. Carbon reasons and works far longer per problem; on BrokenArXiv it produced about 6.7 times as many output tokens as Sol, and its average cost per problem is about 15 times higher across MathArena.
Is MathArena useful for choosing a coding model?
Partly. MathArena now runs models in their native coding harnesses with Python and a time and cost budget, so it reflects agent-style behavior, effort settings and cost per task. But it measures mathematics, not software engineering, so pair it with coding benchmarks such as Terminal-Bench or SWE-bench style tests and your own evaluations.

Share

Keep Reading