Can AI Outsmart Mathematicians? Inside the First AI Math Proof Challenge

In this episode, the host revisits the arc from Deep Blue to today’s generative AI and asks what AI can really do in mathematics. Joe Howlett, Siam’s math reporter, explains how mathematicians think about proofs, why olympiad-style benchmarks aren’t the same as research math, and how an informal group of 11 mathematicians designed a real proof challenge for AI. The podcast reports early results from public models and in‑house systems, notes a thriving online math community testing AI ideas, and debates whether AI’s current approach to math—often brute-force and scaffolded—could transform mathematical discovery or remain a powerful calculator.

Intro: From Deep Blue to Generative AI and Math

The podcast opens by recalling Deep Blue’s 1997 chess victory and asks whether today’s AI, particularly large language models (LLMs), can outthink humans in math. Kendra Pierre‑Louis invites Joe Howlett, Siam’s math reporter, to unpack the math side of AI progress and the types of problems that mathematicians actually care about. The discussion sets up a distinction between textbook problems and genuine research math, where proofs establish statements about the mathematical universe rather than simply producing a right numeric answer.

“AI seems to, at least right now, do math a little differently, and in a way that's a little less impressive to at least some of the mathematicians.” - Mohammad Abu Zayd

What is Math, Really?

Howlett explains that mathematics centers on proving true or false statements about abstract structures, not solving homework questions with checkable answers. He describes how real math involves dealing with objects that may exist in higher dimensions or require proofs that are elegant as well as correct. The host connects this to AI, noting that many AI “wins” in math so far resemble math competition problems or Olympiad-style tasks rather than research breakthroughs, prompting questions about whether AI can contribute to mathematics at a deeper level or simply serve as a calculator.

The First Proof Challenge: Design, Scope, and Guardrails

A group of 11 mathematicians designed a proof challenge to assess how well AI can pose and solve real research math problems. Each participant selected a lemma from an upcoming paper—breaking proofs into smaller theorems—and posed it as a problem for an LLM. Importantly, problems were chosen before any potential training data could include them, ensuring genuine novelty. The aim was to see if an AI could contribute to mathematical research rather than merely reproduce known results. The initial setup sparked a burst of activity in the online math community, with many posted proofs that were mostly garbage but some showed promise. AI teams from OpenAI and Google Gemini attempted the problems, yielding five and six correct solutions respectively, though some of OpenAI’s and Google’s solutions later faced issues.

“Most of these proofs are nonsense, but some of them had some promise.” - Mathematicians 

Early Results and What They Mean for Math

The podcast highlights a striking gap between in‑house AI efforts and publicly available models, with the former achieving higher success rates. A common technique—scaffolding—uses multiple AI systems to interrogate and refine an answer, which appears to boost proof quality and reveal clearer, more robust arguments. The discussion also touches on how AI typically reaches conclusions through a more circuitous, brute-force style rather than inventing new mathematical concepts that distill truth, a process mathematicians sometimes describe as crafting a “beautiful” proof. Some researchers see this as a potential revolution, while others caution that AI may struggle to generate new ideas or abstractions that lead to genuine discoveries.

Looking Ahead: Humans, Tools, and the Pace of Change

As rounds proceed, the first proof team is coordinating with AI companies to establish controls on how problems are approached, ensuring more trustworthy results. The podcast also considers whether iterative rounds can track AI evolution over time and whether AI could ultimately tackle some of mathematics’ biggest open problems. Some mathematicians envision AI helping with discovery, while others emphasize that human curiosity, collaboration, and storytelling will remain central to breakthroughs. The reporter closes by reflecting on the human element of math—the late-night struggles, conference hallway conversations, and the irreplaceable role of human creativity in scientific progress.

“Terminator was a documentary.” - Kendra Pierre-Louis