AI Model Proves Code Safety Automatically, Cutting Verification Costs

The long-standing barrier to software reliability is finally being challenged by artificial intelligence. Traditionally, proving that code runs without errors under all possible conditions was a task reserved for critical systems like aircraft avionics, requiring years of expert labor. Now, Anthropic and its partner Theorem have demonstrated that an AI model can perform this complex mathematical verification almost entirely on its own, potentially transforming how developers trust the code they write.

The core of this new approach involves using Anthropic’s Claude model to mathematically verify existing software rather than generating new code from scratch. In a recent collaboration, the Frontier Red Team at Anthropic worked with Theorem to apply this method to 123 open-source libraries sourced from Google’s OSS-Fuzz security project. The goal was to tackle the high cost and scarcity of formal verification, a field where a team of specialists previously required 20 person-years just to verify the small seL4 operating system kernel.

In just a few weeks, Claude produced approximately 180 million lines of mathematical proof code. The model achieved this by translating crash tests into mathematical problems and analyzing compiled machine code. An independent verification program named Rocq automatically checks these proofs, a process that Anthropic says requires minimal human review. While the researchers describe the generation as fully autonomous, the underlying formal foundations and tools were built by human experts at Theorem.

A significant benefit of this automated process is its ability to uncover defects that conventional testing methods have missed for years. During the verification process, Claude identified hundreds of bugs and was permitted to fix them. Many of these issues only manifest under rare circumstances, such as unusually large input files or specific access sequences. In one notable instance, the team found as many security vulnerabilities in a widely used PDF reader in a single week as are typically reported for that software in an entire year. In another case, a logical error that had gone undetected for over a decade was finally caught.

However, the term formally verified carries specific technical limitations. Anthropic notes that the proofs are valid only under strict conditions, including a specific processor architecture, single-threaded programming, and input sizes under 64 MB. Since these libraries often run on varied hardware and configurations, the verification does not guarantee error-free performance in every real-world scenario. Furthermore, the computational cost remains high, with individual proofs consuming hundreds of millions of tokens, though the total project cost was not disclosed.

Despite these constraints, the project suggests a future where formal verification becomes as standard as basic testing. The process appears to be becoming more efficient over time; a smaller test run indicated that proofs are reusable, and newer models are solving problems with fewer resources. Specifically, Opus 5.5 solved half the proofs using about 40 percent fewer tokens than its predecessor, Opus 5. This efficiency gain could reduce the overall cost of software development by decreasing the time spent on manual code review.

The success of this initiative shifts the focus of AI in software development from mere coding capability to the ability to guarantee safety. If AI can both write code and prove its correctness under defined parameters, the industry may see a major leap in automation. Anthropic expects this method to become as practical as standard testing in the near future, with Theorem’s roadmap already targeting critical infrastructure components like glibc, OpenSSL, and cURL.

This development marks a significant step toward a future where software reliability is mathematically assured rather than statistically tested. By automating the most expensive and labor-intensive part of software assurance, Anthropic and Theorem are positioning AI as a key tool for securing the digital infrastructure that powers modern technology.