Updated on November 4, 2026
Enterprise engineering teams routing production prompts through third-party inference gateways or evaluating pre-release frontier checkpoints face a structural verification blind spot: there is no native way to confirm that a returned token stream was actually generated by the advertised model weights. A cost-cutting API proxy can silently downgrade requests to a smaller quantized model, or a network attacker can tamper with intermediate tokens, and inspecting output quality alone cannot reliably catch the substitution.
To solve this verification gap without exposing proprietary model weights, Anthropic’s Frontier Red Team—working alongside cryptography researchers from EPFL, MIT, StarkWare, and Succinct—has open-sourced FlashProve under the Apache 2.0 license. FlashProve is a zero-knowledge Succinct Non-Interactive Argument of Knowledge (zkSNARK) proof system engineered specifically for large language model inference, allowing a provider to commit cryptographically to a set of private weights and prove that every generated response came from that exact model.
Historically, applying zero-knowledge proofs to neural network forward passes imposed computational overhead several orders of magnitude too slow for multi-billion-parameter Transformers. By using Claude to discover novel optimizations across cryptographic sumcheck protocols and integer-arithmetic prefill pipelines, the researchers achieved proof generation speeds of roughly 110 tokens per second on a single NVIDIA H100 GPU for Llama 4 Maverick, a 400-billion-parameter Mixture-of-Experts (MoE) model running in 4-bit quantized precision.
Because AI-assisted cryptographic optimization can introduce subtle soundness bugs inside complex proof circuits, Anthropic paired the C++ prover and fast verifier with formal mathematical verification in the Lean 4 theorem prover. The open-source release includes executable prover code, both C++ and Lean verifiers, and an interactive theorem visualizer so external security auditors can inspect the top-level soundness guarantees without trusting LLM-generated circuit code blindly.
System & License : FlashProve, released under the Apache 2.0 license by Roy Rinberg, Alex Allain, Alessandro Chiesa, Miranda Christ, Rahul Dalal, Giacomo Fenzi, Ben Mayer, and Guy Weissenberg in the Anthropic Provable Inference report .Benchmark Model : Evaluated on Llama 4 Maverick (400B total parameters, Mixture-of-Experts architecture, 4-bit quantized weights with integer arithmetic in prefill).Proving Throughput : Generates cryptographic zkSNARK proofs at ~110 tokens/second on a single NVIDIA H100 GPU.Proof Size & Compute Overhead : Produces a compact ~19 MB proof per 1,024-token request, requiring 70x to 95x the GPU compute time of unverified vLLM execution—down from multiple orders of magnitude in prior open-source zkML systems.Formal Soundness Auditing : Top-level cryptographic security claims and a reference verifier are formally verified in Lean 4.
How FlashProve Makes 400B-Parameter zkSNARK Proofs Tractable
Why have previous zero-knowledge machine learning (zkML) frameworks struggled to scale past small vision classifiers or 7B models? Standard floating-point matrix multiplications and non-linear activations (SiLU, RMSNorm, and softmax attention) translate poorly into finite-field arithmetic circuits, exploding prover memory and GPU time.
Architectural Metric
Unverified vLLM Baseline
Prior Open-Source zkML Provers
Anthropic FlashProve (H100)
Target Model Scale 400B MoE (Llama 4 Maverick)
Typically <= 7B–13B dense
400B MoE (Llama 4 Maverick)
Weight Precision FP8 / INT4
Fixed-point / custom rings
4-bit quantized + integer prefill
Single-H100 Proving Speed ~8,000–10,000 tok/s (prefill)
< 0.1–1.0 tok/s
~110 tokens/second
Compute Overhead vs. vLLM 1.0x (Reference)
10,000x+ GPU time
70x–95x GPU time
Proof Size (per 1,024 tokens) N/A (Unverified)
Hundreds of MBs to GBs
~19 MB
Formal Verification None
Manual paper proofs
Lean 4 verified top-level theorems
Target Model Scale
Unverified vLLM Baseline 400B MoE (Llama 4 Maverick)
Prior Open-Source zkML Provers Typically <= 7B–13B dense
Anthropic FlashProve (H100) 400B MoE (Llama 4 Maverick)
Weight Precision
Unverified vLLM Baseline FP8 / INT4
Prior Open-Source zkML Provers Fixed-point / custom rings
Anthropic FlashProve (H100) 4-bit quantized + integer prefill
Single-H100 Proving Speed
Unverified vLLM Baseline ~8,000–10,000 tok/s (prefill)
Prior Open-Source zkML Provers < 0.1–1.0 tok/s
Anthropic FlashProve (H100) ~110 tokens/second
Compute Overhead vs. vLLM
Unverified vLLM Baseline 1.0x (Reference)
Prior Open-Source zkML Provers 10,000x+ GPU time
Anthropic FlashProve (H100) 70x–95x GPU time
Proof Size (per 1,024 tokens)
Unverified vLLM Baseline N/A (Unverified)
Prior Open-Source zkML Provers Hundreds of MBs to GBs
Anthropic FlashProve (H100) ~19 MB
Formal Verification
Unverified vLLM Baseline None
Prior Open-Source zkML Provers Manual paper proofs
Anthropic FlashProve (H100) Lean 4 verified top-level theorems
FlashProve closes this gap by combining three architectural choices. First, the provider publishes a cryptographic commitment to the private model weights; the commitment hides the numerical weights completely while assuming the high-level model architecture (layer count, MoE routing topology, and hidden dimensions) is public. Second, prefill operations are executed using exact integer arithmetic over 4-bit quantized weights, avoiding floating-point rounding discrepancies between the prover and verifier. Third, while Claude iteratively optimized the C++ prover routines for H100 tensor and integer pipelines, every top-level protocol reduction was formalized in Lean 4 so that an optimization bug cannot silently allow a forged proof.
Enterprise Use Cases: API Routing Audits and Safety Evaluations
At 70x to 95x compute overhead, cryptographic proving is not yet cheap enough to attach to every casual consumer chat completion. However, 110 tokens per second on a single H100 makes provable inference immediately viable for probabilistic auditing and high-stakes verification workflows:
Third-Party API Router Auditing : Enterprise buyers routing requests through multi-provider aggregators can request cryptographic proofs on a random 1% sample of production requests, mathematically guaranteeing that the router is not substituting a cheaper distilled checkpoint while billing for a frontier model.Pre-Deployment Safety & Red-Team Evaluations : Independent evaluators such as METR, national AI safety institutes, or third-party benchmark operators can verify that the endpoint they are red-teaming is backed by the exact model commitment that will ship to production, preventing benchmark-specific model swapping.High-Integrity Financial & Legal Workflows : Regulated workflows that require non-repudiation can archive the 19 MB proof alongside the prompt and completion transcript, allowing any lightweight CPU verifier to confirm provenance years later without needing GPU hardware or weight access.
Clone and Benchmark the Apache 2.0 Verifier : Pull the FlashProve C++ and Lean 4 verifiers from Anthropic’s open-source release to measure CPU verification latency on 1,024-token proof artifacts (~19 MB).Design Probabilistic Audit Sampling for API Gateways : If your organization consumes frontier LLMs through external model routers or sovereign cloud partners, architect a 0.5%–1.0% statistical sampling pipeline as provable inference endpoints roll out.Pair LLM Code Optimization With Formal Proof Checkers : Adopt Anthropic’s hybrid engineering pattern—using coding agents to optimize low-level C++/CUDA kernels while requiring machine-checked Lean 4 proofs for security-critical invariants.
Beat:
Topics:
# anthropic
# flashprove
# formal-verification
# provable-inference
# zksnarks
