# Anthropic Open-Sources FlashProve for Provable Inference

*Updated on November 4, 2026*

Enterprise engineering teams routing production prompts through third-party inference gateways or evaluating pre-release frontier checkpoints face a structural verification blind spot: there is no native way to confirm that a returned token stream was actually generated by the advertised model weights. A cost-cutting API proxy can silently downgrade requests to a smaller quantized model, or a network attacker can tamper with intermediate tokens, and inspecting output quality alone cannot reliably catch the substitution.

To solve this verification gap without exposing proprietary model weights, Anthropic's Frontier Red Team—working alongside cryptography researchers from EPFL, MIT, StarkWare, and Succinct—has open-sourced FlashProve under the Apache 2.0 license. FlashProve is a zero-knowledge Succinct Non-Interactive Argument of Knowledge (zkSNARK) proof system engineered specifically for large language model inference, allowing a provider to commit cryptographically to a set of private weights and prove that every generated response came from that exact model.

Historically, applying zero-knowledge proofs to neural network forward passes imposed computational overhead several orders of magnitude too slow for multi-billion-parameter Transformers. By using Claude to discover novel optimizations across cryptographic sumcheck protocols and integer-arithmetic prefill pipelines, the researchers achieved proof generation speeds of roughly `110 tokens per second` on a single NVIDIA H100 GPU for `Llama 4 Maverick`, a `400-billion-parameter` Mixture-of-Experts (MoE) model running in 4-bit quantized precision.

Because AI-assisted cryptographic optimization can introduce subtle soundness bugs inside complex proof circuits, Anthropic paired the C++ prover and fast verifier with formal mathematical verification in the Lean 4 theorem prover. The open-source release includes executable prover code, both C++ and Lean verifiers, and an interactive theorem visualizer so external security auditors can inspect the top-level soundness guarantees without trusting LLM-generated circuit code blindly.

<a aria-hidden="true" id="executive-fast-facts"></a>  Fast Facts

- **System &amp; License**: `FlashProve`, released under the `Apache 2.0` license by Roy Rinberg, Alex Allain, Alessandro Chiesa, Miranda Christ, Rahul Dalal, Giacomo Fenzi, Ben Mayer, and Guy Weissenberg in the [Anthropic Provable Inference report](https://www.anthropic.com/research/provable-inference).
- **Benchmark Model**: Evaluated on `Llama 4 Maverick` (`400B` total parameters, Mixture-of-Experts architecture, `4-bit` quantized weights with integer arithmetic in prefill).
- **Proving Throughput**: Generates cryptographic zkSNARK proofs at `~110 tokens/second` on a single `NVIDIA H100` GPU.
- **Proof Size &amp; Compute Overhead**: Produces a compact `~19 MB` proof per `1,024-token` request, requiring `70x to 95x` the GPU compute time of unverified `vLLM` execution—down from multiple orders of magnitude in prior open-source zkML systems.
- **Formal Soundness Auditing**: Top-level cryptographic security claims and a reference verifier are formally verified in `Lean 4`.

## How FlashProve Makes 400B-Parameter zkSNARK Proofs Tractable

Why have previous zero-knowledge machine learning (zkML) frameworks struggled to scale past small vision classifiers or 7B models? Standard floating-point matrix multiplications and non-linear activations (`SiLU`, `RMSNorm`, and softmax attention) translate poorly into finite-field arithmetic circuits, exploding prover memory and GPU time.

| Architectural Metric | Unverified vLLM Baseline | Prior Open-Source zkML Provers | Anthropic FlashProve (H100) |
|---|---|---|---|
| **Target Model Scale** | 400B MoE (`Llama 4 Maverick`) | Typically &lt;= 7B–13B dense | **400B MoE (`Llama 4 Maverick`)** |
| **Weight Precision** | FP8 / INT4 | Fixed-point / custom rings | **4-bit quantized + integer prefill** |
| **Single-H100 Proving Speed** | ~8,000–10,000 tok/s (prefill) | &lt; 0.1–1.0 tok/s | **~110 tokens/second** |
| **Compute Overhead vs. vLLM** | 1.0x (Reference) | 10,000x+ GPU time | **70x–95x GPU time** |
| **Proof Size (per 1,024 tokens)** | N/A (Unverified) | Hundreds of MBs to GBs | **~19 MB** |
| **Formal Verification** | None | Manual paper proofs | **Lean 4 verified top-level theorems** |

FlashProve closes this gap by combining three architectural choices. First, the provider publishes a cryptographic commitment to the private model weights; the commitment hides the numerical weights completely while assuming the high-level model architecture (layer count, MoE routing topology, and hidden dimensions) is public. Second, prefill operations are executed using exact integer arithmetic over 4-bit quantized weights, avoiding floating-point rounding discrepancies between the prover and verifier. Third, while Claude iteratively optimized the C++ prover routines for H100 tensor and integer pipelines, every top-level protocol reduction was formalized in Lean 4 so that an optimization bug cannot silently allow a forged proof.

## Enterprise Use Cases: API Routing Audits and Safety Evaluations

At `70x to 95x` compute overhead, cryptographic proving is not yet cheap enough to attach to every casual consumer chat completion. However, `110 tokens per second` on a single H100 makes provable inference immediately viable for probabilistic auditing and high-stakes verification workflows:

1. **Third-Party API Router Auditing**: Enterprise buyers routing requests through multi-provider aggregators can request cryptographic proofs on a random `1%` sample of production requests, mathematically guaranteeing that the router is not substituting a cheaper distilled checkpoint while billing for a frontier model.
2. **Pre-Deployment Safety &amp; Red-Team Evaluations**: Independent evaluators such as METR, national AI safety institutes, or third-party benchmark operators can verify that the endpoint they are red-teaming is backed by the exact model commitment that will ship to production, preventing benchmark-specific model swapping.
3. **High-Integrity Financial &amp; Legal Workflows**: Regulated workflows that require non-repudiation can archive the `19 MB` proof alongside the prompt and completion transcript, allowing any lightweight CPU verifier to confirm provenance years later without needing GPU hardware or weight access.

  Next Steps

1. **Clone and Benchmark the Apache 2.0 Verifier**: Pull the FlashProve C++ and Lean 4 verifiers from Anthropic's open-source release to measure CPU verification latency on `1,024-token` proof artifacts (`~19 MB`).
2. **Design Probabilistic Audit Sampling for API Gateways**: If your organization consumes frontier LLMs through external model routers or sovereign cloud partners, architect a `0.5%–1.0%` statistical sampling pipeline as provable inference endpoints roll out.
3. **Pair LLM Code Optimization With Formal Proof Checkers**: Adopt Anthropic's hybrid engineering pattern—using coding agents to optimize low-level C++/CUDA kernels while requiring machine-checked Lean 4 proofs for security-critical invariants.