# Moonshot AI Releases Kimi K3.1 Pro 2.8T MoE Open Weights

*Updated on November 4, 2026*

Moonshot AI has escalated the open-weight frontier model race by releasing the full model weights for Kimi K3.1 Pro on Hugging Face. Scaling to `2.8 trillion` total parameters with `104 billion` activated parameters per token, Kimi K3.1 Pro is a natively multimodal, agentic Mixture-of-Experts (MoE) model featuring a `1,048,576-token` (`1M`) context window and quantization-aware training (`MXFP4` weights with `MXFP8` activations) out of the box.

Post-trained from the base Kimi K3 checkpoint using large-scale joint reinforcement learning across hundreds of coding, 3D generation, financial research, and visual reasoning domains, Kimi K3.1 Pro matches or surpasses top proprietary US frontier models on autonomous software engineering benchmarks. On the grueling `Terminal-Bench 4.0` evaluation, Kimi K3.1 Pro scores `65.7%`, outperforming both `GPT-6 Astra` (`58.2%`) and `Claude Fable 5.1` (`57.9%`) and finishing within `0.7` points of `Claude Opus 5.5` (`66.4%`).

Even more striking is its endurance on long-horizon engineering tasks: on `SWE Marathon v1.1`, Kimi K3.1 Pro achieves `51.7%`, beating `Claude Opus 5.5` (`46.3%`), `Claude Fable 5.1` (`45.6%`), and `GPT-6 Astra` (`42.5%`). On `BenchCAD` (without external tools), it posts `72.8%` compared to `60.6%` for Claude Fable 5.1, demonstrating strong spatial and 3D programmatic modeling capabilities.

To make a `2.8T` parameter model with a `1M` context window practical to serve in production, Moonshot AI overhauled the attention stack from Kimi K3—replacing full-attention Gated Multi-Head Latent Attention (MLA) layers with `24 Gated DeepSeek Sparse Attention (Gated DSA)` layers paired with `69 KDA` layers, while training the weights natively in microscaling `MXFP4` precision.

<a aria-hidden="true" id="executive-fast-facts"></a>  Fast Facts

- **Architecture &amp; Parameter Scale**: `2.8 trillion` total parameters (`104B` active per token) across `93 layers` (`1` dense layer, `896` routed experts with `16` active + `2` shared experts per token), published on the [Moonshot AI Kimi K3.1 Pro Hugging Face repository](https://huggingface.co/moonshotai/Kimi-K3.1-Pro).
- **Hybrid Sparse Attention Stack**: Combines `69 KDA` layers with `24 Gated DeepSeek Sparse Attention (Gated DSA)` layers (`32` index heads, `128` index head dimension, `Top-K = 2048`) to support a `1,048,576-token` context window.
- **Native Multimodal Vision &amp; Precision**: Integrates a `401M-parameter` `MoonViT-V2` vision encoder, `SiTU-GLU` activations, and native Quantization-Aware Training (`MXFP4` weights / `MXFP8` activations).
- **Terminal-Bench 4.0**: Scores `65.7%` (at `max` reasoning effort), ahead of `GPT-6 Astra` (`58.2%`), `Claude Fable 5.1` (`57.9%`), `DeepSeek-V4.1-Flash-1017` (`42.3%`), and `GLM-5.3` (`41.8%`).
- **SWE Marathon v1.1 &amp; BenchCAD**: Leads all evaluated proprietary and open models on `SWE Marathon v1.1` (`51.7%` vs. `46.3%` for Claude Opus 5.5) and scores `72.8%` on `BenchCAD w/o tools`.

## Architectural &amp; Benchmark Breakdown: Kimi K3.1 Pro vs. Frontier Peers

How does Kimi K3.1 Pro process `1,048,576` tokens without suffering quadratic KV-cache blowups during multi-hour coding marathons? By replacing full-attention Gated MLA layers with `Gated DSA` (`Top-K = 2048` token selection across `32` index heads), the model bounds long-context attention compute while preserving fine-grained retrieval across massive codebases and financial filings.

| Benchmark / Spec | Kimi K3.1 Pro (Open Weights) | Claude Opus 5.5 (Proprietary) | GPT-6 Astra (Proprietary) | Claude Fable 5.1 (Proprietary) | DeepSeek-V4.1-Flash |
|---|---|---|---|---|---|
| **Total / Active Parameters** | **2.8T / 104B (MoE)** | Proprietary | Proprietary | Proprietary | MoE (Open/API) |
| **Context Window** | **1,048,576 tokens** | 200K–1M tokens | 256K–1M tokens | 200K–1M tokens | 128K–256K tokens |
| **Terminal-Bench 4.0** | **65.7%** | **66.4%** | 58.2% | 57.9% | 42.3% |
| **SWE Marathon v1.1** | **51.7% (1st)** | 46.3% | 42.5% | 45.6% | 40.0% |
| **Terminal-Bench-Science 0.1** | 38.1% | 63.3% | **68.1%** | 40.0% | 15.7% |
| **BenchCAD (w/o Tools)** | **72.8%** | — | — | 60.6% | — |
| **Serving Precision** | **MXFP4 / MXFP8 QAT** | Proprietary | Proprietary | Proprietary | FP8 |

## Why Native MXFP4 Quantization-Aware Training Matters for Self-Hosting

At `2.8 trillion` parameters, an unquantized `BF16` checkpoint would require `5.6 TB` of aggregate GPU HBM—demanding at least five 8-GPU `NVIDIA B200` or `H200` nodes just to load the weights. Because Moonshot AI performed Quantization-Aware Training (QAT) targeting **MXFP4 weights and MXFP8 activations**, the entire `2.8T` checkpoint compresses to roughly `1.4 TB`. That allows enterprise infrastructure teams to serve Kimi K3.1 Pro on a single 8x `B200` (`1.44 TB` HBM3e) or dual 8x `H200` node via `vLLM` or `Transformers` without post-training quantization accuracy loss.

  Next Steps

1. **Test Kimi K3.1 Pro on Long-Horizon SWE Tasks**: Evaluate `Kimi K3.1 Pro` via the Moonshot API (`platform.kimi.ai`) or self-hosted `vLLM` endpoints on multi-hour repository refactoring tasks where it leads `SWE Marathon v1.1` (`51.7%`).
2. **Leverage Native MXFP4/MXFP8 Support on Blackwell &amp; Hopper**: Deploy the `compressed-tensors` checkpoint directly on `NVIDIA B200` or `H200` clusters to cut VRAM footprint by `75%` relative to `BF16` with zero post-training perplexity degradation.
3. **Integrate MoonViT-V2 for UI-to-Code &amp; 3D CAD Pipelines**: Combine Kimi K3.1 Pro's `401M` native vision encoder and `72.8%` `BenchCAD` performance for automated frontend screenshot-to-code and parametric 3D asset generation.