Tracking platforms—including the Artificial Analysis Index, SWE-bench Verified, and BenchLM—have refreshed their official rankings following major model releases across OpenAI, Anthropic, Google, and Zhipu AI. The updated standings outline a specialized competitive landscape where no single model holds universal dominance.
While Anthropic’s Claude Fable 5.1 and Claude Mythos 5 lead in complex software engineering and codebase refactoring, OpenAI’s GPT-6 Astra resets efficiency expectations by slashing per-task coding spend. Meanwhile, open-weight international models are capturing decisive leads in domain-specific workflows, led by Zhipu AI’s dominance in end-to-end data science.
Model
Artificial Analysis
SWE-bench Verified
BenchLM (Data)
Cost / SWE-bench Task
Claude Mythos 5 100 / 100 (#1 Overall)
76.1% (Leader)
94.2%
$0.65
Claude Fable 5.1 100 / 100 (Tied #1)
74.2%
92.8%
$0.38
GPT-6 Astra 61 / 100 (Preliminary)
73.8% (Preliminary)
91.4%
$0.20 (Efficiency King)
GLM-4.7 54 / 100
68.4%
96.8% (Leader)
$0.14
Gemini 3.8 Flash 48 / 100 (#1 Value)
58.2%
84.5%
$0.06
Muse Spark 1.3 42 / 100
51.4%
78.2%
$0.02
Claude Mythos 5
Artificial Analysis 100 / 100 (#1 Overall)
SWE-bench Verified 76.1% (Leader)
BenchLM (Data) 94.2%
Cost / SWE-bench Task $0.65
Claude Fable 5.1
Artificial Analysis 100 / 100 (Tied #1)
SWE-bench Verified 74.2%
BenchLM (Data) 92.8%
Cost / SWE-bench Task $0.38
GPT-6 Astra
Artificial Analysis 61 / 100 (Preliminary)
SWE-bench Verified 73.8% (Preliminary)
BenchLM (Data) 91.4%
Cost / SWE-bench Task $0.20 (Efficiency King)
GLM-4.7
Artificial Analysis 54 / 100
SWE-bench Verified 68.4%
BenchLM (Data) 96.8% (Leader)
Cost / SWE-bench Task $0.14
Gemini 3.8 Flash
Artificial Analysis 48 / 100 (#1 Value)
SWE-bench Verified 58.2%
BenchLM (Data) 84.5%
Cost / SWE-bench Task $0.06
Muse Spark 1.3
Artificial Analysis 42 / 100
SWE-bench Verified 51.4%
BenchLM (Data) 78.2%
Cost / SWE-bench Task $0.02
76.1% SWE-bench Verified Peak: Claude Mythos 5 holds the record at 76.1%, followed closely by Claude Fable 5.1 at 74.2%.$0.20 per Issue Coding Efficiency: GPT-6 Astra posted a preliminary 73.8% SWE-bench resolve rate while consuming 47% less spend per solved task than Claude Fable ($0.38).96.8% BenchLM Data Science Record: Zhipu AI’s GLM-4.7 leads global data science evaluations, outperforming US frontier models on pandas, SQL, and statistical regressions.100/100 Composite Ratings: Claude Fable 5.1 and Mythos 5 retain top marks on Artificial Analysis quality indices, with Astra tied on raw capability.Top API Value Champion: Gemini 3.8 Flash secures the #1 rating for throughput-to-cost ratio across high-volume production APIs. In-Depth Benchmark Analysis
SWE-bench Verified: Codebase Resolution Versus Token Spend
SWE-bench Verified remains the gold standard for evaluating an AI model’s real-world software engineering capability. Unlike simple function-level coding tests like HumanEval, SWE-bench presents models with actual GitHub issues from complex repositories (such as Django, SymPy, and scikit-learn) requiring the agent to edit multiple files, write regression tests, and pass CI builds.
Claude Fable 5.1 demonstrated superior multi-file consistency, successfully resolving 74.2% of problems. However, OpenAI Astra’s preliminary 73.8% score has sparked intense industry discussion because of its radical cost advantage. While Fable agents averaged $0.38 per resolution attempt, Astra completed the test battery at ~$0.20 per issue. For enterprise development platforms running hundreds of thousands of automated issue evaluations monthly, Astra represents a massive budget efficiency gain.
BenchLM: Zhipu AI Claims the Data Science Lead
The most surprising result on the leaderboards came in BenchLM, which evaluates end-to-end data science pipelines: ingesting raw CSV/Parquet files, performing exploratory data analysis, writing clean vectorized pandas/numpy transforms, and executing statistical regressions.
Zhipu AI’s GLM-4.7 claimed the #1 position with a 96.8% success score, beating both Claude Mythos 5 (94.2%) and GPT-6 Astra (91.4%). GLM-4.7 demonstrated an ability to identify data leakage in training splits and automatically optimize SQL query execution plans, establishing itself as the premier model for automated business intelligence.
Artificial Analysis Index: Composite Rankings Shift
The Artificial Analysis Index provides a holistic, multi-dimensional view of model performance by weighting coding, reasoning, knowledge recall, latency, and context recall:
Claude Fable 5.1 & Mythos 5: Awarded 100/100 composite scores, representing the pinnacle of unconstrained frontier intelligence.GPT-6 Astra: Entered at 61/100 on preliminary evaluations, with scores heavily dragged down by its strict gated cyber availability and slower time-to-first-token during deep chain-of-thought execution.Gemini 3.8 Flash: Awarded 48/100 overall, but earned the #1 “Best Value” distinction due to its sub-140ms latency and 180 token/second throughput.
Map Workloads to Strengths: Route multi-file software engineering tasks to Claude Fable 5.1 or Astra, while directing tabular data analysis and SQL generation to GLM-4.7.Calculate Full Lifecycle Token Costs: Measure cache hit rates and thinking token volume rather than evaluating models on sticker price alone.Test Localized Fine-Tunes: Benchmark open-weight models like GLM-4.7 within your private cloud environment to evaluate on-premises compliance advantages against hosted APIs. Updated on September 5, 2026
