
Watch TensorLean video (opens in a new tab)
Ask an LLM a question in English, and Thai characters can suddenly appear in its answer. Ask it to write code, and the generated program contains obvious syntax errors.
Some Claude users encountered these problems in late August 2025, when a TPU runtime optimization disrupted next-token computations. The system kept running, even though its responses had already gone wrong. In October 2026, Google DeepMind researchers reported two more numerical bugs in core AI operations: Tiny rounding errors could erase the signal a model learns from, or create a false signal where there should be none. In their experiments, this was enough to push a model off target when it was trained to imitate another model.
These silent errors carry significant stakes: if a $100 million AI training run required just 1% of its computation to be repeated due to errors, that would mean an additional $1 million in compute costs. With this in mind, how can developers rigorously verify that code computes correctly before running it at scale?
On November 20, researchers at CertiK and Columbia University announced a breakthrough with TensorLean, marking the world’s first formal verification of production AI kernels. The team has applied verification to more than 100 production GPU kernels, including kernels from DeepSeek and the widely used FlashAttention library.
Beyond Testing: Ruling Out Bugs Within a Defined Scope
Kernels are small programs running on GPUs. They perform much of the underlying computation involved in training models and generating responses. TensorLean aims to provide machine-checkable mathematical proofs for these programs.
Conventional testing looks for problems by running examples. However, tests can only check the cases developers think to write. A bug can hide in an edge case nobody anticipated, such as an unusual sequence length, a rare batch size, or different interleaving of concurrent requests. Passing common tests does not prevent an error from appearing at a different scale or configuration.
Formal verification provides a more rigorous approach. Researchers first express the intended computation, applicable conditions, and permitted numerical error as precise mathematical requirements, and then prove that the program consistently satisfies them.
Once established, the proof covers every input and execution that satisfies the stated conditions, ruling out errors and bugs within the verified scope. This is a central value of formal verification: it can help uncover code that violates requirements and, when a proof succeeds, provide a mathematical guarantee that the corresponding errors cannot occur.
The Breakthrough: Verifying Code Built for Speed
Real-world AI computing code often relies on complex optimizations to run faster. Depending on the task, GPU programs may change how data is arranged in memory and how work is divided across threads. They overlap data transfers with computation to reduce waiting. They also use lower-precision, faster arithmetic to increase throughput.
These techniques improve speed, but make correctness harder to guarantee. Does the program compute correctly across different task sizes? Are data transfers coordinated correctly? Does the error introduced by lower precision always remain within the permitted tolerance?
“These complex optimizations are precisely what make production AI code fast,” said Wei Qiang and Ganxiang Yang, TensorLean research team members. “Our challenge was to complete the proofs without leaving out those real execution conditions, making rigorous verification applicable to high-performance code in actual use.”
TensorLean’s technical breakthrough is to bring these conditions together in automated verification. It directly works on the compiled GPU code (the PTX representation) so that the proof addresses the specific high-performance implementation.
All proofs are developed in Lean and are machine-checkable. Every step is checked by a foundational prover that accepts only valid reasoning. If a proof passes, its logic is sound and trustworthy.
The research has already produced concrete findings: the team evaluated 149 compiled GPU kernel instances, proved that 83 met their Lean specifications, and uncovered 38 bugs across six libraries, including 2 in DeepSeek’s DeepGEMM and 23 in FlashAttention.
These findings show that even widely used, performance-optimized code can contain bugs that conventional testing has not uncovered. After identifying a problem, developers can repair the code, rerun verification, and obtain correctness proofs for implementations that meet the requirements.
Continuous Assurance for Rapid AI Optimization
TensorLean provides a rigorous verification method that can be run repeatedly.
LLM developers continually optimize underlying computing code to accelerate training and reduce inference costs. Changing data layouts, adjusting execution configurations, or adopting lower-precision arithmetic can improve speed, but each change can also introduce computational errors that conventional testing may not detect.
Teams first specify the intended computation and applicable conditions. After optimizing the code, they check whether the new implementation still meets those requirements. When the intended computation and underlying assumptions remain unchanged, the same specification can be reused for new optimized versions. If an optimization changes the input conditions or numerical error requirements, the specification must be adjusted accordingly.
For developers, this means checking “Does it still compute correctly?” alongside “How much faster is it?”, thus creating an opportunity to find computational errors before large-scale training and deployment.
“After finding a bug, we still need to prove that the repaired program meets its requirements under defined conditions,” said Ronghui Gu, co-founder of CertiK and a computer science professor at Columbia University. “For large-model developers, every performance improvement should be backed by reliable evidence of correctness. We want rigorous verification to keep pace with code changes and reduce the risk of computational errors reaching large-scale training and deployment.”
In October, Anthropic and Theorem also published research on using Claude to construct mathematical proofs for open-source software (opens in a new tab). TensorLean focuses on AI’s own underlying computations, demonstrating the practical value of automated formal verification for production AI code.
We're extending our formal verification expertise to the computing foundations of AI. To learn more about our broader formal verification capabilities and services, explore our Formal Verification page.