Moonshot AI Releases Kimi K3.1 Pro 2.8T MoE Open Weights

Updated on November 4, 2026

Moonshot AI has escalated the open-weight frontier model race by releasing the full model weights for Kimi K3.1 Pro on Hugging Face. Scaling to 2.8 trillion total parameters with 104 billion activated parameters per token, Kimi K3.1 Pro is a natively multimodal, agentic Mixture-of-Experts (MoE) model featuring a 1,048,576-token (1M) context window and quantization-aware training (MXFP4 weights with MXFP8 activations) out of the box.

Post-trained from the base Kimi K3 checkpoint using large-scale joint reinforcement learning across hundreds of coding, 3D generation, financial research, and visual reasoning domains, Kimi K3.1 Pro matches or surpasses top proprietary US frontier models on autonomous software engineering benchmarks. On the grueling Terminal-Bench 4.0 evaluation, Kimi K3.1 Pro scores 65.7%, outperforming both GPT-6 Astra (58.2%) and Claude Fable 5.1 (57.9%) and finishing within 0.7 points of Claude Opus 5.5 (66.4%).

Even more striking is its endurance on long-horizon engineering tasks: on SWE Marathon v1.1, Kimi K3.1 Pro achieves 51.7%, beating Claude Opus 5.5 (46.3%), Claude Fable 5.1 (45.6%), and GPT-6 Astra (42.5%). On BenchCAD (without external tools), it posts 72.8% compared to 60.6% for Claude Fable 5.1, demonstrating strong spatial and 3D programmatic modeling capabilities.

To make a 2.8T parameter model with a 1M context window practical to serve in production, Moonshot AI overhauled the attention stack from Kimi K3—replacing full-attention Gated Multi-Head Latent Attention (MLA) layers with 24 Gated DeepSeek Sparse Attention (Gated DSA) layers paired with 69 KDA layers, while training the weights natively in microscaling MXFP4 precision.

Architectural & Benchmark Breakdown: Kimi K3.1 Pro vs. Frontier Peers

How does Kimi K3.1 Pro process 1,048,576 tokens without suffering quadratic KV-cache blowups during multi-hour coding marathons? By replacing full-attention Gated MLA layers with Gated DSA (Top-K = 2048 token selection across 32 index heads), the model bounds long-context attention compute while preserving fine-grained retrieval across massive codebases and financial filings.

Why Native MXFP4 Quantization-Aware Training Matters for Self-Hosting

At 2.8 trillion parameters, an unquantized BF16 checkpoint would require 5.6 TB of aggregate GPU HBM—demanding at least five 8-GPU NVIDIA B200 or H200 nodes just to load the weights. Because Moonshot AI performed Quantization-Aware Training (QAT) targeting MXFP4 weights and MXFP8 activations, the entire 2.8T checkpoint compresses to roughly 1.4 TB. That allows enterprise infrastructure teams to serve Kimi K3.1 Pro on a single 8x B200 (1.44 TB HBM3e) or dual 8x H200 node via vLLM or Transformers without post-training quantization accuracy loss.