# First Proof Exam Highlights AI Math Performance Disparity Proprietary frontier models dominate research-level mathematics while open-weight alternatives struggle to keep pace. - **Source:** [Scientific American Content: Global](https://brifing.si/source/4f066134-e06d-4415-87ef-305e3dc2aba6) - **Published:** 2026-10-14 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/597dd687-8397-4838-ae50-74b928539ab5 ## The First Proof Benchmark The First Proof project provides an independent evaluation of AI models in research-level mathematics. Launched in February 2026, the initiative aims to cut through the noise of imperfect benchmarks. By utilizing expert-selected problems and mock peer reviews, the team offers a rigorous assessment of how LLMs handle complex, high-stakes mathematical reasoning tasks. ## Frontier Versus Open Models Results from the third batch of testing reveal a significant performance gap between proprietary frontier models and open-weight alternatives. While subscription-based models demonstrate exceptional capabilities in solving research problems, open-source models currently lag far behind. This disparity suggests that advanced mathematical reasoning is becoming a pay-to-play domain for top-tier research applications. ## Performance Metrics and Growth The top-performing model, ProofCouncil, successfully solved nine out of ten research problems in the latest round. This represents a rapid ascent compared to February, where the top model solved only a few problems. By June, the winner managed six or seven, highlighting the dramatic acceleration of AI capabilities throughout the year 2026. ## The Open-Weight Struggle Despite the progress seen in proprietary systems, open-weight models like Kimi K3 failed to solve any of the ten research problems in the latest evaluation. Mathematicians involved in the study noted that these models fell flat, emphasizing an urgent need for the community to invest more resources into improving open-source mathematical reasoning capabilities. ## Methodology and Peer Review The First Proof team solicited ten problems from experts across various subdisciplines. Models were required to answer without human intervention, ensuring an objective test of their reasoning. Three graders evaluated each submission using a mock peer-review process, determining if the answers were acceptable for publication in a peer-reviewed academic journal. ## Privacy and Data Concerns Many mathematicians are seeking alternatives to frontier models due to privacy concerns and potential data leakage. Researchers are increasingly paranoid about using closed systems for sensitive work, leading to requests for reliable open-source options. The First Proof team is actively testing these alternatives to help the scientific community find secure, transparent research tools. ## Transparency and Model Access The First Proof rules require entrants to be openly available, which excluded OpenAI's latest blockbuster model from the evaluation. This lack of transparency prevents outside researchers from conducting direct performance comparisons. Without access to the internal workings of frontier models, the scientific community remains unable to fully verify the claims surrounding these breakthroughs. ## Future of Mathematical Research AI models have become so proficient at proving well-defined statements that future testing of frontier models on this specific task may lose relevance. As LLMs approach the ability to supersede human mathematicians, the focus must shift toward ensuring these powerful tools remain accessible, transparent, and secure for the broader global research community. ## Sources covering this story - https://www.scientificamerican.com/article/new-first-proof-exam-reveals-how-ai-powered-math-is-a-pay-for-play-sport/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._