# LiteLLM Suspects GitHub Token Leak in AI Gateway Logs Behind Malicious PyPI Package Release

LiteLLM, one of the most widely deployed open-source AI gateway libraries used to route API calls across OpenAI, Anthropic, Google Gemini, and AWS Bedrock, has disclosed a severe supply-chain compromise that pushed a credential-stealing remote access trojan (RAT) to PyPI, Docker Hub, and GitHub Container Registry (GHCR). According to the maintainers' post-mortem investigation, the attackers likely gained release access after a privileged GitHub token was inadvertently exposed inside AI gateway logs.

For engineering teams running AI applications in production, the incident is a wake-up call about two compounding risks: unpinned Python/Docker dependencies in automated CI/CD pipelines, and verbose debug logging in AI proxy servers. On October 21, 2026, at approximately 4:00 a.m. PST (11:39 a.m. UTC), attackers bypassed release controls to publish `litellm` version `1.107.0` on PyPI alongside the `v1.107.0`, `latest`, `main-latest`, and `main-stable` container tags on Docker Hub and GHCR.

Any developer workstation, Kubernetes pod, or CI runner that pulled an unpinned `litellm` package between 11:39 a.m. and 1:23 p.m. UTC (or pulled the affected Docker images between 11:46 a.m. and 2:20 p.m. UTC) downloaded a dual-stage payload. Without requiring the user to import or call LiteLLM in their code, the poisoned package used a `.pth` startup hook and `sitecustomize.py` to harvest browser, cloud, and AI API secrets and establish an hourly remote code execution loop.

Within 4.5 hours of the initial exfiltration, cloud security researchers observed the attackers actively testing stolen AWS instance role credentials against Amazon Bedrock endpoints—checking which foundation models were enabled so they could hijack enterprise cloud quotas in an attack pattern known as "LLMjacking."

## Fast Facts: The LiteLLM v1.107.0 Supply-Chain Compromise

- **Compromised Artifacts**: `litellm==1.107.0` on PyPI, plus `v1.107.0`, `latest`, `main-latest`, and `main-stable` container images on Docker Hub and GHCR, as detailed in [The Hacker News' security breakdown](https://thehackernews.com/2026/10/litellm-suspects-github-token-leak-in.html).
- **Exposure Window (Oct 21, 2026)**: 11:39 a.m. to 1:23 p.m. UTC on PyPI (1 hour 44 minutes) and 11:46 a.m. to 2:20 p.m. UTC on Docker registries (2 hours 34 minutes).
- **Suspected Root Cause**: Exposure of a privileged GitHub release token inside AI gateway logs, allowing attackers to compromise release pipeline controls.
- **Dual-Loader Execution Mechanism**: First documented wild attack combining Python path configuration (`cook.pth` $\\rightarrow$ `cook.py` running `privcollect`) with `sitecustomize.py` persistence on Linux and macOS.
- **Exfiltration & C2 Indicators**: Harvested credentials were POSTed to `litellm[.]dev/healthcheck`, while an hourly background daemon (`/tmp/remote_code_loader.lock`) polled `github[.]com/letters9813` (`pythonclient-test/main/domain.txt`) for `/codepy` payloads.
- **Post-Exploitation LLMjacking**: Starting 4.5 hours after exfiltration (4:20 p.m. to 5:01 p.m. UTC), attackers from `168.100.8.57` and `165.227.201.8` ran `sts:GetCallerIdentity` and `bedrock:ListFoundationModels` to hijack victim AI compute budgets.

## Anatomy of the Attack: Dual Python Startup Hooks & Cloud LLMjacking

Security researchers at Wiz (Benjamin Read and Rami McCarthy) and Aikido Security (Ilyas Makari) analyzed the poisoned `1.107.0` build and noted that it executes automatically the moment the Python interpreter starts in an environment where the package is installed.

| Attack Stage | Malicious File / Indicator | Technical Behavior on Victim Machine | What Defenders Should Check |
|---|---|---|---|
| **Stage 1: Credential Harvester** | `cook.pth` & `cook.py` (`privcollect` routine) | Processed automatically by Python on startup; scrapes environment variables, cloud keys, browser data, and AI API tokens | Outbound HTTP POST requests to `litellm.dev/healthcheck` |
| **Stage 2: Hourly RAT Daemon** | `sitecustomize.py` & `/tmp/remote_code_loader.lock` | Forks a background process on Linux/macOS that polls a GitHub dead-drop (`letters9813/pythonclient-test`) every 60 minutes | Presence of `/tmp/remote_code_loader.lock` and requests to `/codepy` |
| **Stage 3: Cloud & AI Enumeration** | IP `168.100.8.57` & `165.227.201.8` | Runs `sts:GetCallerIdentity`, `bedrock:ListFoundationModels`, `ListModels`, and IAM discovery (`ListRoles`, `ListAccessKeys`) | AWS CloudTrail logs between 11:39 UTC and 18:00 UTC on Oct 21 |
| **Entry Vector: Gateway Logs** | Verbose proxy/gateway debug logs | Unredacted `Authorization` headers or CI environment variables written to centralized log sinks | Audit AI gateway logging configs for raw bearer tokens and PATs |

## Why AI Gateways Are Prime Targets for Supply-Chain Attackers

An AI gateway sits at the exact intersection of every secret an attacker wants. A single LiteLLM instance typically holds API keys for OpenAI, Anthropic, Azure OpenAI, and Vertex AI, alongside AWS IAM role credentials for Bedrock, database connection strings for spend tracking, and internal developer tokens.

### 1. The Hidden Danger of Verbose AI Gateway Logging

When debugging tool-calling failures, streaming timeouts, or MCP header propagation, developers frequently enable verbose request logging (`--detailed_debug` or `LITELLM_LOG=DEBUG`). If an AI gateway logs raw HTTP headers, environment dumps, or full tool-call payloads without strict token redaction, any GitHub Personal Access Token (PAT) or cloud secret passed through the environment can end up stored in plaintext log files or third-party observability dashboards.

### 2. Why `latest` Docker Tags and Unpinned `pip install` Are Fatal in CI/CD

The PyPI exposure window lasted just 104 minutes, and GitHub took down the attacker's dead-drop repository around 1:30 p.m. UTC before the second-stage `domain.txt` file was populated with a live payload beyond initial test values. Even so, automated CI/CD runners and autoscaling Kubernetes clusters that ran `pip install litellm` (without a version pin and SHA-256 hash) or pulled `litellm:main-latest` during that two-hour window immediately surrendered their AWS instance roles via Stage 1 (`privcollect`).

## Practical Remediation Checklist for Engineering & Security Teams

1. **Search for Compromise Artifacts Immediately**:

- Check all Python virtual environments, container layers, and CI runners for `litellm==1.107.0`, `cook.pth`, `cook.py`, or the lockfile `/tmp/remote_code_loader.lock`.
- Query DNS and firewall logs for outbound connections to `litellm.dev/healthcheck` or `raw.githubusercontent.com/letters9813/`.

1. **Audit AWS CloudTrail for LLMjacking Activity**:

- Search CloudTrail logs from October 21 onward for `sts:GetCallerIdentity`, `bedrock:ListFoundationModels`, `iam:ListRoles`, or `iam:ListAccessKeys` originating from `168.100.8.57` or `165.227.201.8`.

1. **Rotate All Keys Present on Affected Nodes**:

- Because `privcollect` scraped entire environment variable tables and local credential stores, rotate every OpenAI, Anthropic, AWS, GitHub, and database secret that was mounted on any machine that built or ran during the exposure window.

1. **Pin Package Versions with Hashes & Redact Gateway Logs**:

- Replace floating Docker tags (`latest`, `main-stable`) with immutable image digests (`@sha256:...`), enforce `--require-hashes` in `pip` installations, and enable automatic secret scrubbing in your AI gateway logging pipeline.