Z.ai GLM-Legal Scores 19.2% All-Pass on Harvey LAB, Topping Claude Fable 5 at $0.15 Per Task

Domain-specific post-training and co-evolved agent harnesses are outperforming general-purpose frontier models on zero-tolerance professional workflows at a fraction of the inference cost. Z.ai has unveiled GLM-Legal, a specialized legal engineering system built on its natively multimodal GLM-5.3-Flash Mixture-of-Experts backbone (320 billion total parameters with 18 billion active parameters per token), achieving an independently verified 19.17% all-pass rate and 92.5% individual criterion pass rate on Harvey’s open-source Legal Agent Benchmark (Harvey LAB).

Evaluated independently by Vals AI across 79 competing models and agent systems, GLM-Legal ranked #7 overall at an average inference cost of just $0.15 per test. That score nearly triples the 6.67% all-pass baseline of the underlying GLM-5.3-Flash model (+12.5 percentage points) and decisively beats larger general-purpose models including Moonshot AI’s Kimi K3 (12.92%), Anthropic’s Claude Fable 5 (11.25%), and OpenAI’s GPT-6 Astra (5.42%), while trailing Muse Spark 1.2 (25.42%) at the top of the leaderboard.

Why do top-tier frontier models score between 5% and 25% on Harvey LAB when their individual rubric accuracy exceeds 90%? Unlike multiple-choice bar exam benchmarks, Harvey LAB consists of 1,250 multi-document legal workflows spanning 24 practice areas—including M&A due diligence, credit agreement redlining, patent claim charting, and cross-border regulatory schedules—graded against more than 75,000 lawyer-authored rubric criteria (roughly 60 strict criteria per task). Under Vals AI’s all-pass metric, a model receives credit for a task only if it satisfies 100% of the ~60 rubric requirements without a single omitted covenant, miscalculated share count, or formatting defect.

In addition, Harvey LAB disables live internet access during evaluation and requires the agent to natively manipulate binary office files—reading and generating .docx redlines, .xlsx capitalization tables, and .pptx board summaries—within a strict 30-minute per-task execution timeout. General-purpose models frequently fail not on legal doctrine, but on multi-file cross-referencing or broken document serialization.

As reported in MarkTechPost’s technical breakdown of Z.ai GLM-Legal on Harvey LAB, Z.ai closed that gap without inflating active parameter count by combining hybrid sparse and linear attention with mid-training on hundreds of billions of legal and financial tokens and a co-evolved agent harness trained via reinforcement learning to verify cross-document citations before writing final .docx and .xlsx outputs.

Fast Facts: Z.ai GLM-Legal & Harvey LAB Benchmark

• Base Architecture: Built on GLM-5.3-Flash (320B total parameters, 18B active parameters, hybrid sparse + linear attention).
• Benchmark Scope (Harvey LAB): 1,250 real-world legal tasks across 24 practice areas graded against 75,000+ expert criteria (~60 per task) by Vals AI with internet access disabled.
• GLM-Legal Performance: 19.17% all-pass score (#7 of 79 systems) and 92.5% individual criterion pass rate at $0.15 average cost per task.
• Domain Post-Training Lift: +12.5 percentage points over base GLM-5.3-Flash (6.67%)—a 2.87x improvement in zero-defect task completion.
• Head-to-Head Frontier Results: Outperformed Kimi K3 (12.92%), Claude Fable 5 (11.25%), and GPT-6 Astra (5.42%); trailed Muse Spark 1.2 (25.42%).

Harvey LAB Leaderboard Comparison (Vals AI Audit, Oct 2026)

Why Co-Evolved Agent Harnesses Beat Raw Parameter Scaling

The +12.5-point jump from GLM-5.3-Flash (6.67%) to GLM-Legal (19.17%) demonstrates that wrapping a frontier model in a generic prompt template is no longer sufficient for high-stakes enterprise document work. When a task requires checking 60 interdependent criteria across a 150-page credit agreement and three Excel schedules, a single unverified tool call causes the entire task to fail the all-pass gate.

By co-evolving the model weights alongside the docx, pptx, and xlsx tool execution harness during reinforcement learning, GLM-Legal learns to allocate test-time compute toward self-auditing its generated spreadsheets and redlines against the prompt’s implicit rubric before terminating the session—delivering superior reliability at $0.15 per run.

The 4-Step Enterprise Implementation Playbook

1. Adopt All-Pass Rubric Grading Over Average Accuracy: Stop evaluating internal legal, financial, and compliance AI pilots on mean criterion accuracy (where 92.5% masks an 80.8% task failure rate); require 100% rubric all-pass scoring on representative 50-criterion workflows before authorizing production deployment.
2. Co-Design File Manipulation Tools With Model Checkpoints: Equip legal and finance agents with deterministic Python libraries (python-docx, openpyxl, python-pptx) paired with an automated verification loop that re-reads the rendered binary file to catch broken tables or missing clauses prior to human review.
3. Route High-Volume Due Diligence to 18B-Active MoE Specialists: At $0.15 per complex multi-document task, deploy domain-specialized sparse MoE models like GLM-Legal for first-pass contract extraction and schedule reconciliation, reserving human attorneys for the final 7.5% of edge-case criteria.
4. Enforce Offline Sandbox Execution for Privileged Matters: Mirror Harvey LAB’s zero-internet evaluation architecture by running contract analysis agents inside network-isolated VPC sandboxes with zero external web egress.