Benchmarks · Analysis
MathArena Leaderboard: Gemini 4 Carbon Takes #1
Gemini 4 Carbon now leads MathArena at 85.4%, ahead of Claude Fable 5.2 and GPT-6.1 Sol. It costs about 15 times more per problem than Sol despite identical token prices, and its lead over Fable sits inside the confidence intervals.
Metarix Editorial · Nov 15, 2026 · 9 min read
- MathArena
- Gemini 4 Carbon
- Claude Fable
- GPT-6.1 Sol
- Kimi K3.1 Pro
- Antigravity CLI
- Codex CLI
- Claude Code
- Google DeepMind
- Anthropic
- OpenAI
- Moonshot AI
- Benchmarks & Deep Dives
- AI Agents & Coding
Documentation-based research

On this page
Gemini 4 Carbon is now the top model on MathArena, the ETH Zurich and INSAIT benchmark built from fresh research-level math problems. In the November 13, 2026 update it scored 85.4% expected performance, ahead of Claude Fable 5.2 at 82.8% and GPT-6.1 Sol at 80.6%, ending a long run of OpenAI models in first place.
The more useful number for builders is in the next column. Carbon costs $15.70 per problem on average, Fable 5.2 costs $40.20, and GPT-6.1 Sol costs $1.07, even though Carbon and Sol charge exactly the same per token. Here is what the new leaderboard actually shows, why the per-problem costs are so far apart, and how to use it when you pick a model for hard reasoning work.
Quick verdict
Gemini 4 Carbon has the best MathArena score, but its lead over Fable 5.2 sits inside the confidence intervals, and it buys about 5 points over GPT-6.1 Sol for roughly 15 times the cost per problem. For most hard-reasoning workloads, keep GPT-6.1 Sol (or another cheap, strong model) as the default and escalate to Carbon only for the problems the default fails, with wall-clock and budget caps in place. Budget for Carbon's per-token price doubling on December 1.
Scroll the table horizontally on smaller screens.
| Model | Expected performance | Expected cost per problem | Setting and harness used |
|---|---|---|---|
| Gemini 4 Carbon | 85.4% ±3.0% | $15.70 ±$1.33 | High effort, Antigravity CLI, Google API |
| Claude Fable 5.2 | 82.8% ±5.3% | $40.20 ±$2.59 | Max effort, Claude Code, via OpenRouter (Anthropic only) |
| GPT-6.1 Sol | 80.6% ±2.7% | $1.07 ±$0.018 | Max effort, Codex, via OpenRouter (OpenAI only) |
| Best open model: Kimi K3.1 Pro | 54.3% ±5.8% | $3.70 ±$0.27 | Max effort, Kimi harness, via OpenRouter |
What changed on the MathArena leaderboard
MathArena added results for Claude Fable 5.2 and Gemini 4 Carbon on November 13, according to the update log on its home page. Carbon went straight to first place overall, Fable 5.2 to second, and GPT-6.1 Sol, which had led, dropped to third. MathArena co-author Jasper Dekoninck called it the first time "in a very long time" that a non-OpenAI model held the top spot.
The rest of the top ten is OpenAI and Anthropic: GPT-6 Astra (max) at 78.0%, GPT-6 Sol (max) at 75.9% and Claude Opus 5.5 (high) at 75.3% follow, per the full model list. Google's own Gemini 4 Argon sits 12th at 62.1%, which shows how much Carbon's longer reasoning changes the result within the same model family. The best open-weight model, Kimi K3.1 Pro, is 15th at 54.3%.
What MathArena measures
MathArena uses new problems so that models can't have seen them during training. The overall score currently leans on research-level sets built from arXiv papers published in the past month. ArXivMath asks a model to rederive a precise result from a recent paper. BrokenArXiv hands it a plausible but false statement, often a conjecture a new paper has just refuted, and asks for a proof; full marks go only to models that say the statement is false. ArXivLean asks for formal proofs in Lean. If you are new to reading boards like this, our guide to reading LLM leaderboards covers the general traps.
The lead is narrower than the ranking suggests
MathArena publishes confidence intervals, and they matter here. Carbon's 85.4% ±3.0 spans about 82.4% to 88.4%. Fable 5.2's 82.8% ±5.3 spans about 77.5% to 88.1%, so the two overlap almost entirely. Carbon against Sol (77.9% to 83.3%) overlaps only slightly. On MathArena's own numbers, first and second place are statistically close to a tie; first and third are a narrow but real gap.
Coverage also differs. Carbon's model page lists nine results, including ArXivLean and three months each of ArXivMath and BrokenArXiv. Fable 5.2's page lists four: ArXivMath and BrokenArXiv for September and October only. MathArena fills missing results with an item-response model, which is why Fable's interval is wider. Its ranking could move as more results come in.
Scroll the table horizontally on smaller screens.
| Competition | Gemini 4 Carbon | Claude Fable 5.2 | GPT-6.1 Sol |
|---|---|---|---|
| ArXivMath 10/2026 | 93.62%, $5.29 | 93.62%, $22.28 | 90.43%, $1.21 |
| ArXivMath 09/2026 | 96.23%, $6.04 | 96.23%, $24.13 | 91.51%, $0.74 |
| BrokenArXiv 10/2026 | 82.72%, $18.06 | 73.46%, $39.78 | 80.25%, $1.30 |
| BrokenArXiv 09/2026 | 82.39%, $18.60 | 80.50%, $33.44 | 82.70%, $1.15 |
| ArXivLean 09/2026 (formal proofs) | 62.26%, $45.11 | No result listed | 35.85%, $1.16 |
Read across the rows and the picture is consistent. On ArXivMath the three models are within a few points. On BrokenArXiv, Sol matches or nearly matches Carbon at a fraction of the cost. Carbon's clearest lead is formal proof writing in Lean, where it scored 62.26% against Sol's 35.85% on the one set both attempted, the only row where the gap is clearly larger than the noise.
Same price per token, 15 times the price per problem
Gemini 4 Carbon and GPT-6.1 Sol have identical list prices today: $2 per million input tokens, $0.10 per million cached input tokens and $10 per million output tokens, per Google's Gemini API pricing and OpenAI's pricing page. Prices as of Nov 15, 2026. Yet on MathArena Carbon costs $15.70 per problem and Sol $1.07, about 15 times more. Fable 5.2, at $8 input and $40 output per million tokens on Anthropic's pricing page, costs $40.20, about 38 times Sol.

With equal rates, the whole gap is token volume. On BrokenArXiv overall, MathArena reports 539,577 output tokens for Carbon against 80,413 for Sol, about 6.7 times more, while the cost gap on that set is about 14 times. Our reading is that the rest comes from input and cached context re-sent as the agent works across many turns in its harness; MathArena doesn't break input tokens out on the model pages. The Decoder, citing Dekoninck, reports that Carbon's most common failure is running out of time and that it almost never gives up, in one case launching a script that ran for more than eleven hours without checking on it.
What the price change on December 1 does
Google's pricing page lists Carbon's current rates as valid through November 30, 2026. From December 1 they double to $4 input, $0.20 cached input and $20 output per million tokens. Because every rate doubles, the same MathArena runs would cost about $31.40 per problem, roughly 29 times GPT-6.1 Sol at its current price. We covered the schedule in our Gemini 4 Carbon pricing analysis.
Scroll the table horizontally on smaller screens.
| Model | Per problem | Per 1,000 problems | Expected performance |
|---|---|---|---|
| GPT-6.1 Sol (max) | $1.07 | $1,070 | 80.6% |
| Gemini 4 Carbon (high), through Nov 30 | $15.70 | $15,700 | 85.4% |
| Gemini 4 Carbon (high), from Dec 1 | about $31.40 | about $31,400 | 85.4% |
| Claude Fable 5.2 (max) | $40.20 | $40,200 | 82.8% |
Reasoning effort is a cost dial, not a free upgrade
MathArena's list includes some models at more than one effort setting, and the results are a warning against defaulting to max. Claude Fable 5.1 at high effort scored 73.6% for $21.67 per problem; at max it scored 72.3% for $74.72, more than three times the cost for no gain. GPT-6 Astra at low effort scored 73.3% for $1.07, against 78.0% for $4.03 at max.
The top three were not run at the same setting either: Carbon at high, Fable 5.2 and Sol at max. MathArena's methodology post notes it moved Fable 5.1 from max to high because it regularly spent more than 512,000 output tokens without a single tool call. So these are best-recommended configurations, not like-for-like effort comparisons. If you deploy any of these models, test at least two effort levels on your own tasks before picking one.
How MathArena runs the models, and why agent builders should care

Since September 2026, MathArena runs each model in the coding harness it ships with: the Antigravity CLI for Gemini, Codex for OpenAI models and Claude Code for Claude. Each attempt runs in a Docker container with Python and SageMath but no internet, under a 12-hour time limit and a $100 model-cost budget, four runs per problem. The exact harness versions and settings are published on each model page, and the evaluation code is open source.
That makes MathArena closer to an agent benchmark than an old-style question-and-answer test. The models write and run scripts, check intermediate results and decide when to stop. Three lessons carry over directly to coding and research agents:
- Cap wall-clock time and spend per task. MathArena needed both limits after runs got out of hand. Carbon's main failure mode is timing out, so a hard timeout is a correctness feature, not just a cost control.
- Reward "this is wrong" answers. BrokenArXiv gives only partial credit to a model that silently repairs a false statement, and none to one that "proves" it. That is the same failure as a coding agent that edits a test until it passes. If your evals only check that a task completed, add cases where the right answer is to refuse or flag the request.
- Measure cost per solved task, not per token. Two models with identical rate cards ended up 15 times apart. Our cost per successful task guide shows how to track this in production.
A routing pattern that uses the leaderboard
The numbers point to an escalation setup: try the cheap, strong model first, and pass only failures or low-confidence answers to the expensive one. MathArena's own research found that routing compute by problem difficulty cut cost by 29% at equal accuracy for Lean theorem provers on PutnamBench, per its dynamic compute allocation post. The sketch below is illustrative; we did not run it for this article, and you need your own verifier, such as tests, a checker script or a judge model.
# Illustrative escalation router (pseudo-Python)ATTEMPTS = [ {"model": "gpt-6.1-sol", "effort": "high", "timeout_s": 1800, "max_usd": 3}, {"model": "gemini-4-carbon", "effort": "high", "timeout_s": 10800, "max_usd": 40},]def solve(task): for cfg in ATTEMPTS: result = run_agent(task, **cfg) # your harness: Codex, Antigravity CLI, etc. if result.ok and verify(task, result): # tests, checker or judge return result return flag_for_human(task)
At MathArena's averages, if the first model solves most problems, the expensive model only runs on the remainder, so your blended cost stays close to the cheap model's while the hardest cases get Carbon-level effort. Set the escalation budget with the December 1 price in mind.
Which model for which job
Scroll the table horizontally on smaller screens.
| If you need | Start with | Why |
|---|---|---|
| Hard math or quantitative reasoning at volume | GPT-6.1 Sol | 80.6% for $1.07 per problem; within a few points of the leaders on most sets |
| The highest score on the hardest problems, cost secondary | Gemini 4 Carbon | #1 overall and a clear lead on formal Lean proofs; price doubles Dec 1 |
| A Claude-based stack for long agentic work | Claude Fable 5.2, tested against Opus 5.5 | Close to Carbon on score but the most expensive per problem; Opus 5.5 scored 75.3% for $3.92 |
| Open weights or self-hosting | Kimi K3.1 Pro | Best open model at 54.3%, far below the closed leaders on this benchmark |
| Coding agents specifically | Check coding benchmarks too | MathArena is math; see Terminal-Bench and SWE-bench style results before choosing |
Verdict
Gemini 4 Carbon deserves its first place: it is the strongest model on MathArena's fresh research problems and the clear leader on formal proofs. But the leaderboard's cost column is the real story. GPT-6.1 Sol delivers about 94% of Carbon's score for about 7% of the cost per problem at today's prices, and Claude Fable 5.2 costs the most without a statistically clear edge over Carbon (see our Fable 5.2 pricing analysis). Use Sol or a similar model as the default, escalate the failures to Carbon under strict time and spend caps, and rerun the math once Carbon's December 1 price takes effect.
Sources
- MathArena leaderboard and update log (Nov 13, 2026 update, accessed Nov 15, 2026)
- MathArena: all models and model pages for Gemini 4 Carbon, Claude Fable 5.2, GPT-6.1 Sol and Kimi K3.1 Pro (accessed Nov 15, 2026)
- MathArena: ArXivMath and BrokenArXiv, harder problems and revised grading (Sep 2026)
- MathArena: BrokenArXiv and dynamic compute allocation for Lean provers
- MathArena evaluation code on GitHub
- Jasper Dekoninck on X: Gemini 4 Carbon takes #1 (Nov 13, 2026)
- Google: Gemini Developer API pricing, OpenAI: API pricing and GPT-6.1 Sol model page, Anthropic: Claude API pricing (accessed Nov 15, 2026)
- The Decoder: Gemini 4 Carbon takes the MathArena crown but costs 15 times more than GPT-6.1 Sol (Nov 14, 2026)
Tools covered in this article
Tool
MathArena
Benchmarks & Deep Dives · Free
Tool
Gemini 4 Carbon
AI Agents & Coding · Paid
Tool
Claude Fable
AI Agents & Coding · Paid
Tool
GPT-6.1 Sol
AI Agents & Coding · Paid
Tool
Kimi K3.1 Pro
AI Agents & Coding · Paid
Tool
Antigravity CLI
AI Agents & Coding · Freemium
Tool
Codex CLI
AI Agents & Coding · Freemium
Tool
Claude Code
AI Agents & Coding · Paid
Frequently asked questions
- What is MathArena?
- MathArena is a benchmark platform from the SRI Lab at ETH Zurich and INSAIT that tests language models on newly released math problems, including research-level questions drawn from recent arXiv papers, so models can't have memorized them. It reports each model's expected performance with confidence intervals and its average cost per problem, and its evaluation code is open source.
- Which model is #1 on MathArena?
- As of the November 13, 2026 update, Gemini 4 Carbon (high effort) is first with 85.4% expected performance, followed by Claude Fable 5.2 (max) at 82.8% and GPT-6.1 Sol (max) at 80.6%. The confidence intervals of the top two overlap heavily, so their order is not statistically settled.
- Why does Gemini 4 Carbon cost more than GPT-6.1 Sol on MathArena if the token prices are the same?
- Both models list $2 per million input tokens and $10 per million output tokens as of November 15, 2026, so the difference is how many tokens each uses. Carbon reasons and works far longer per problem; on BrokenArXiv it produced about 6.7 times as many output tokens as Sol, and its average cost per problem is about 15 times higher across MathArena.
- Is MathArena useful for choosing a coding model?
- Partly. MathArena now runs models in their native coding harnesses with Python and a time and cost budget, so it reflects agent-style behavior, effort settings and cost per task. But it measures mathematics, not software engineering, so pair it with coding benchmarks such as Terminal-Bench or SWE-bench style tests and your own evaluations.