For the first three years of the generative AI boom, infrastructure spending was overwhelmingly concentrated in centralized hyperscale training clusters. Enterprises tolerated 300-millisecond cross-continental network round trips because most early AI features were asynchronous chatbots where users expected to wait a few seconds for text to stream back.
In late 2026, two shifts have broken that centralized model. First, real-time voice, e-commerce, and interactive workflow agents require sub-50-millisecond time-to-first-token close to the end user. Second, modern autonomous agents do not just output text—they write and execute Python, JavaScript, and shell code on the fly, creating a massive remote-code-execution risk if that untrusted code runs inside an enterprise’s core application servers.
This week, content delivery and cloud security giant Akamai moved to capture both workloads simultaneously. The Nasdaq-listed infrastructure provider announced Akamai Inference —a managed, OpenAI-compatible AI inference platform backed by newly expanded Nvidia Blackwell GPU clusters connected to more than 4,350 global edge locations —alongside Akamai Agent Sandbox , a dedicated isolation environment engineered specifically to contain and execute code generated by autonomous AI agents.
By pairing open-weight model inference (spanning 12 models from 8 billion to 2.8 trillion parameters) with edge code sandboxing and web application security on a single network fabric, Akamai is pitching enterprise CIOs a compelling alternative to stitching together separate GPU clouds, container sandboxes, and API gateways.
12 Open-Weight Models (8B to 2.8T Parameters): At launch, Akamai Inference hosts 12 open-weight models plus RAG embedding models via OpenAI-compatible APIs, including Nvidia Nemotron 3 Ultra , OpenAI gpt-oss-120b , and Google Gemma 4 .Flexible Deployment Tiers: Enterprises can choose between serverless pay-per-use inference for rapid feature deployment or dedicated GPU deployments for full infrastructure control and data residency.Akamai Agent Sandbox: A purpose-built restricted execution environment that isolates AI agents and the dynamic code they write from core enterprise networks and host infrastructure.Blackwell Ultra Today, Vera Rubin Next Year: Akamai expanded its GPU fleet with Nvidia RTX Pro 6000 Blackwell Server Edition GPUs (optimized for low-latency inference) and Nvidia Blackwell Ultra GPU clusters (for dedicated self-hosted frontier models), while preparing its data centers for Nvidia Vera Rubin hardware in 2027.4,350+ Edge Locations: Akamai Cloud connects its core GPU and CPU regions to more than 4,350 global edge locations , minimizing network transit latency and wrapping inference endpoints in Akamai’s existing DDoS, WAF, and API security stack. Why Open-Weight Inference Is Moving to the Network Edge
As Adam Karon, Chief Operating Officer and GM of Akamai Cloud, noted during the launch, enterprise spending has decisively tilted from model training toward production inference and agentic execution. More importantly, the maturation of open-weight models in late 2026—ranging from lightweight 8B models and Google’s Gemma 4 to medium-weight reasoning workhorses like OpenAI’s gpt-oss-120b2.8-trillion-parameter mixture-of-experts architectures like Nvidia Nemotron 3 Ultra —means enterprises no longer need to route every API call to a closed hyperscaler endpoint.
When an enterprise switches from a closed proprietary API to an OpenAI-compatible open-weight endpoint hosted on Akamai Inference, it gains three operational advantages:
Infrastructure Dimension
Centralized Closed-Model API
DIY Self-Hosted Cloud GPUs
Akamai Inference + Agent Sandbox
Model Portability & Range Locked to single vendor’s closed weights
Full control, but high MLOps maintenance
OpenAI-compatible API across 12 models (8B to 2.8T ) + custom weights
Hardware Tiers Shared multi-tenant black box
Long reservation queues for latest GPUs
Serverless pay-per-use OR dedicated RTX Pro 6000 Blackwell / Blackwell Ultra
Network & Edge Proximity Centralized regional data centers
Single-region VPC deployment
Core GPU clusters routed across 4,350+ global edge locations
Agent Code Execution Requires separate 3rd-party sandbox vendor
Manual Kubernetes/gVisor setup required
Built-in Akamai Agent Sandbox + native WAF/API security
Model Portability & Range
Centralized Closed-Model API Locked to single vendor’s closed weights
DIY Self-Hosted Cloud GPUs Full control, but high MLOps maintenance
Akamai Inference + Agent Sandbox OpenAI-compatible API across 12 models (8B to 2.8T ) + custom weights
Hardware Tiers
Centralized Closed-Model API Shared multi-tenant black box
DIY Self-Hosted Cloud GPUs Long reservation queues for latest GPUs
Akamai Inference + Agent Sandbox Serverless pay-per-use OR dedicated RTX Pro 6000 Blackwell / Blackwell Ultra
Network & Edge Proximity
Centralized Closed-Model API Centralized regional data centers
DIY Self-Hosted Cloud GPUs Single-region VPC deployment
Akamai Inference + Agent Sandbox Core GPU clusters routed across 4,350+ global edge locations
Agent Code Execution
Centralized Closed-Model API Requires separate 3rd-party sandbox vendor
DIY Self-Hosted Cloud GPUs Manual Kubernetes/gVisor setup required
Akamai Inference + Agent Sandbox Built-in Akamai Agent Sandbox + native WAF/API security
First, API compatibility without vendor lock-in : because Akamai Inference exposes standard OpenAI-compatible SDK endpoints, engineering teams can swap a workload from a proprietary model to gpt-oss-120b or Nemotron 3 Ultra by changing a single base URL and model string.
Second, hardware right-sizing : Akamai split its GPU expansion into two distinct hardware profiles. Workloads that need ultra-fast token generation on 8B–120B parameter models run on Nvidia RTX Pro 6000 Blackwell Server Edition GPUs , whereas regulated enterprises running massive 2.8T-parameter models or proprietary fine-tunes can provision dedicated Blackwell Ultra clusters—with a clear upgrade path to Nvidia’s Vera Rubin architecture next year.
Solving the “Untrusted Code” Problem With Akamai Agent Sandbox
The most practical component of Akamai’s announcement for software architects is Akamai Agent Sandbox . Over the past six months, engineering teams have learned the hard way that an LLM is far more accurate at math, data transformation, CSV parsing, and chart generation when it writes and executes a quick Python script rather than trying to predict numbers token-by-token.
However, giving an LLM agent a live code interpreter inside your production cloud VPC is a security nightmare. As Jon Alexander, SVP of Cloud Technology at Akamai, emphasized: “Every team running agents needs a boundary around the code those agents write.”
If an attacker slips an indirect prompt injection into an uploaded spreadsheet, webpage, or support ticket, a naive code-executing agent will happily write a script that scans internal subnet ports, reads environment variables containing AWS or database credentials (process.env), or exfiltrates customer records. By coupling Akamai Agent Sandbox directly adjacent to Akamai Inference , an agent can generate code on a Blackwell GPU and execute that code milliseconds later inside a strictly fenced, ephemeral sandbox sitting behind Akamai’s edge firewall—completely air-gapped from the customer’s internal production database and IAM credentials.
For CTOs and platform engineers planning their 2027 AI infrastructure roadmap, Akamai’s entry alongside Cloudflare’s edge worker ecosystem signals that inference + code sandboxing + API security is converging into a single edge-cloud layer.
The 4-Step Enterprise Implementation Playbook
Benchmark gpt-oss-120b and Gemma 4 Against Closed API Costs: Identify your top three highest-volume production prompts currently running on proprietary frontier APIs. Since Akamai Inference uses OpenAI-compatible APIs, run a shadow traffic benchmark against serverless gpt-oss-120b, Gemma 4, and Nemotron 3 Ultra to measure latency at the edge and calculate your monthly token savings.Never Execute Agent-Generated Code in Application Containers: Audit every workflow where an agent writes Python, Bash, or SQL/JavaScript. Immediately strip local exec() or standard Docker container execution out of your core application cluster and route all agent-generated scripts into an isolated runtime boundary like Akamai Agent Sandbox (or hardware-isolated microVMs) with zero access to internal VPC metadata endpoints.Match GPU SKU to Model Parameter Footprint: Stop overpaying for multi-node HBM clusters when running mid-sized models. Route 8B to 70B/120B real-time inference to Nvidia RTX Pro 6000 Blackwell Server Edition instances for optimal cost-per-token and low latency, and reserve dedicated Blackwell Ultra clusters for trillion-parameter models (Nemotron 3 Ultra) or strict single-tenant compliance workloads.Place WAF and Rate-Limiting at the Edge Before the GPU: Protect your inference budget from token-exhaustion DDoS attacks and automated scraping by terminating client connections at the edge (across Akamai’s 4,350 POPs) and enforcing per-user token rate limits before a request ever wakes up a dedicated Blackwell GPU.
Next Steps
