Benchmarks & Deep Dives · Free
MathArena
MathArena is a public benchmark platform that evaluates large language models on newly released mathematics problems, so that scores are not inflated by problems the models may have seen during training. It is run by the SRI Lab at ETH Zurich and INSAIT, is supported by a grant from Epoch AI, and publishes its leaderboards at matharena.ai. The evaluation code is open source under the MIT license on GitHub (eth-sri/matharena), and datasets are published on Hugging Face.
Who it is for: developers, researchers and technical buyers who want an independent, contamination-resistant view of how well frontier and open-weight models reason, and what that reasoning costs. Because every result is reported with an average cost per problem in US dollars, it is also useful for comparing price against capability rather than looking at accuracy alone.
What it measures: MathArena runs several competition families, including final-answer contests such as AIME and HMMT, proof-based olympiads such as USAMO and IMO, visual math, Project Euler, and research-level sets built from recent arXiv papers. ArXivMath asks models to rederive precise results from papers published in the past month. BrokenArXiv gives models plausible but false statements, often refuted conjectures, and checks whether a model notices the statement is false instead of producing a confident fake proof. ArXivLean and LeanLean test formal proofs in the Lean proof assistant.
How models are run: since September 2026, models are run in their own recommended coding harnesses, for example the Antigravity CLI for Gemini, Codex for OpenAI models and Claude Code for Claude, inside a Docker sandbox with Python and SageMath but no internet access. Each attempt has a 12-hour time limit and a $100 model-cost budget, and each model runs four times per problem. Answers are graded by an LLM judge or a rubric, and an overall "expected performance" score combines observed results with an item-response-theory estimate for missing ones, with confidence intervals.
Key features: model pages with per-competition accuracy, rank, cost and output tokens; the exact sampling parameters and harness versions used; raw model traces; and a compare view across models.
Pricing type: free to browse; the code and data are free to reuse under their licenses.
Alternatives: Epoch AI's benchmarking hub, Artificial Analysis, the LMArena leaderboards, FrontierMath, and Humanity's Last Exam for broad reasoning.
Official site: https://matharena.ai
