LiteLLM 1.107.0 Supply-Chain Compromise: Keys Stolen on Python Startup, and the Gateway at the Center

First, the boundary: this is not a rehash of old news

This site has mentioned LiteLLM twice before, and neither was this story. One was an October 7 product story about Moyai, LiteLLM's open-source cloud coding agent, built around its claim of costing 79% less than Devin. The other was a single line inside a roundup of runtime incidents, flagging the LiteLLM poisoning only as a lead. This article is the full incident deep-dive: it covers only the poisoned LiteLLM 1.107.0 release of October 21, 2026 — how it got into the release pipeline, how it stole credentials the moment you started Python, and what to do afterwards. If you want just one sentence, here it is: if you installed 1.107.0, do not rush to delete and reinstall; the first step is to stop running any Python in that environment.

A two-hour window put the poisoned package on PyPI and Docker

On the morning of October 21, 2026, LiteLLM's release pipeline was compromised and the malicious 1.107.0 release was pushed to PyPI and container registries. The official timeline is precise: the seven wheels on PyPI lived from 11:39:51 to 13:23 UTC; on Docker Hub, the v1.107.0, latest, main-latest and main-stable tags pointed at the poisoned image index from 11:46:56 UTC until 14:20 UTC, when they were removed or repointed to clean images; the same tags on GHCR were unavailable by 13:51 UTC. Wiz gives a slightly different figure, saying PyPI quarantined the release at 13:20 UTC — a difference of minutes that does not change the conclusion. It was a short window measured in hours, and it was long enough. Citing ClickPy data, Wiz reports roughly 49,000 downloads of the malicious version. Note carefully: that is a download count, not a victim count. No party has published a confirmed number of environments that actually executed the package or of credentials actually exfiltrated.

One clarification from the official post deserves emphasis: the official v1.107.0 source tag on GitHub did not contain the malicious files. The poisoned code came from an attacker-controlled lookalike repository, and the official signing pipeline was hijacked into publishing the poisoned wheels. Reading the source repository or checking the version number alone would not have revealed the problem; the compromise sat in the build and release step. That is why the official guidance keeps repeating the safe version — 1.106.2, or any explicit version published before the incident — rather than a floating tag such as latest.

The most insidious part: you did not need to import LiteLLM to be hit

The power of this poisoning is not how deeply it hid, but how early it fired. The malicious wheel dropped three files straight into the root of the Python environment's site-packages directory: cook.pth, cook.py and sitecustomize.py. When the Python interpreter starts, site initialization processes executable lines in .pth files and then tries to import the sitecustomize module — all before your application imports anything at all.

There were two mutually reinforcing startup chains. The first starts with the single line "import cook" in cook.pth, which loads cook.py; cook.py decodes an embedded Base64- and zlib-compressed program and runs a credential collector that calls itself PRIVCOLLECT, usually as a background daemon thread. Microsoft's analysis notes that an ordinary Windows invocation of Python to install an unrelated dependency activated the collector — the victim did not need to be using the LiteLLM gateway at all. The second chain is a Unix-oriented loader in sitecustomize.py, which detaches into the background, takes a lock file, and periodically reads a GitHub-hosted domain.txt file — a mutable dead-drop pointer. If the pointer resolves to a server, the loader is designed to fetch code from it and execute it directly. But the caveat matters: at the time researchers examined it, domain.txt was empty, and second-stage retrieval and execution were not confirmed. This is a re-armable backdoor, not a confirmed second-stage attack, and the two should not be conflated.

What it stole: nearly every credential your machine can name

Microsoft's and Wiz's teardowns of the collector agree closely, and the list is uncomfortably long. On the cloud and workload side: AWS, Azure CLI and Google Cloud configuration and credential caches, Kubernetes service-account files, Docker configuration, mounted secrets, and AWS and GCP instance metadata services. On the developer-tool side: Git credentials, package-manager configuration, dotenv files, private-key files, and authentication files for AI assistants and IDEs, including GitHub Copilot, VS Code and MCP integrations. On the browser side: credential-related data and browsing history from Chromium-family and Firefox browsers, plus shell, database-client and interactive-tool histories. On the process-environment side, the worst detail: on Linux it read other processes' environments through procfs, so a secret did not even need to belong to the LiteLLM process — if a process on the same host could see it, it could be swept up too.

For the LiteLLM gateway itself, the attacker added a targeted kill shot. When its prerequisites were met, the collector spawned a Python child process that used Prisma to read two proxy database tables, 500 rows each, targeting the provider credentials and model configuration stored in the proxy. Wiz describes the query as selecting directly from the LiteLLM credentials table with a 500-row limit. In other words, the gateway product's selling point — centralizing every model provider's keys behind one proxy — became its largest blast radius in this incident: one stolen gateway could expose every provider key it proxied.

Microsoft also documented the exfiltration in detail. Each run generated a fresh 32-byte AES key and a 16-byte initialization vector, encrypted the collection with AES-256-CBC, wrapped the AES key with an embedded 4,096-bit RSA public key, and packed the result into a gzip archive with three members — data.bin, key.bin and hdr.json — sent by HTTP POST to the attacker-controlled lookalike domain www[.]litellm[.]dev, at its healthcheck path. That domain mimics LiteLLM's legitimate infrastructure; do not visit or copy it unmodified anywhere.

The attack chain: three engineers' GitHub CLI tokens and a near-identical repo name

The attack chain reconstructed in LiteLLM's October 21 evening post reads like a case study in permission management gone wrong. First, release permissions had been temporarily widened on the Monday of that week so two internal engineers could publish daily bugfix backports. The attacker already held GitHub CLI OAuth tokens belonging to three developers, one of which could run GitHub workflows. An initial attempt to modify the release workflow and point the releaser at a different source repository was blocked by branch protection; the attacker then used a token belonging to one of the organization's owners to disable that protection and force-push the change, used another developer's credential to pass the production approval gate, and finally ran the modified workflow pointed at the lookalike repository Berri-AI/litellm — note the hyphen; the legitimate organization is BerriAI, with no hyphen — so the poisoned artifacts were published through the official signing flow. Afterwards, a third developer's credential was used to delete the production and staging workflow records, obscuring the investigation trail.

How were the tokens lost in the first place? The official account still carries qualifiers such as "we believe" and "we suspect," and there is no final verdict yet. LiteLLM observed that in specific situations Claude Code embeds a user's GitHub CLI OAuth token into a response body, and where response bodies are logged, those tokens can appear in LiteLLM logs; it also found an innocuously named account in its own internet-exposed internal LiteLLM deployment, and believes the attacker reached the logged tokens through that deployment, possibly via a recently patched vulnerability. Microsoft's October 23 post states plainly that it has not independently determined how the attacker first accessed the upstream environment or obtained those credentials. Attribution is unconfirmed, and any claim pinning this incident to a specific group or country goes beyond the evidence.

There is, however, genuinely observed post-exploitation activity. Wiz reports that roughly 4.5 hours after exfiltration, between 16:20 and 17:01 UTC on October 21, the attacker began validating AWS instance-role credentials harvested from compromised CI runners and Kubernetes nodes, calling sts:GetCallerIdentity followed by bedrock ListFoundationModels calls — a pattern consistent with LLMjacking, stealing cloud credentials to burn model calls. A small number of environments also saw IAM enumeration and general discovery calls. That proves some stolen credentials were tried; it does not prove every downloader was compromised, and the scale of loss remains unknown.

If you installed it in the window: stop first, then clean up in order

The official and Microsoft guidance share a counterintuitive premise: do not start by running python or pip inside the suspect environment "to check." Any Python start re-executes cook.pth and sitecustomize.py, effectively re-triggering the collector yourself. The right order is to isolate first — stop the relevant Python processes and automation so they cannot restart — and then inspect the suspect environment from outside, from a known-clean machine with trusted tools, rather than investigating from inside it.

Next comes cleanup and rebuilding. The official advice is to delete cook.pth, cook.py, sitecustomize.py and their cached bytecode from every affected site-packages directory, kill the detached daemon holding the lock file, and reinstall pinned to litellm==1.106.2; rebuilding the virtual environment or image from scratch is better, because the startup files live outside the litellm application directory, so deleting only that directory leaves the hooks behind. Microsoft goes further: find every affected copy across installed environments, pip and uv caches, internal mirrors, lock files, build outputs and container images, and compare artifacts by hash — using container digests rather than floating tags to establish exactly what ran.

The final and most painful step is rotating every credential the suspect environment could reach. The official list includes the instance or task IAM role, every provider key in the proxy database plus master and virtual keys, the DATABASE_URL password, and any secret present in the environment of any process on the host. Microsoft adds CI/CD secrets, source-control and registry tokens, SSH and GPG keys, AI coding-tool tokens and browser sessions, and warns that rotating a key does not automatically revoke an issued session — active sessions must be revoked explicitly. Check CloudTrail for GetCallerIdentity calls from the instance role around install time, and for any later use of that role's credentials from outside your own infrastructure. Enterprise customers on the private registry were not affected, the official post states clearly — there is no need to panic on that front.

A judgement for vibe coders: what this incident really changes

My judgement is that this incident redefines the AI gateway category from a convenience tool into a high-value target. Previous supply-chain stories were about an obscure package being poisoned or a dependency name being squatted. This time the poisoned package sits on the path of every model call for many teams, and a gateway by design concentrates every upstream key, database connection string and tenant policy in one place. An attacker did not need to break your application code; breaking the gateway's release pipeline was enough to collect a whole batch of teams' cloud credentials and model keys at once. That "single point of concentration, single point of failure" structure is especially common in vibe coding: one person wires five or six model providers through one gateway for convenience, putting the entire estate in one basket.

Several practices should therefore move from "recommended" to "default." First, pin everything by version and hash: exact versions in pip installs, container digests instead of latest — latest and main-latest were precisely the tags hit hardest here. Second, separate the gateway from the development machine: do not run a production gateway holding the full set of provider keys on the same workstation where you constantly experiment and install packages, and isolate CI runners and Kubernetes nodes with elevated privileges all the more, since Wiz's first observed credential validations came from exactly those environments. Third, layer your secrets: the cloud role used by the gateway should carry minimum privilege, not simultaneous access to the database, cloud APIs and every provider key, or the rotation scope after an incident becomes the whole company. Fourth, admit packages and images to production only through an internal mirror that has reviewed them, rather than letting workstations and CI pull floating versions straight from public sources.

There is a deeper lesson hiding in the start of the attack chain. The suspected token-leak path runs through an AI coding tool embedding a credential into a response body that then got logged — in other words, the AI toolchain itself is becoming a new credential-exposure surface. When you let an AI tool investigate a problem, read logs or run commands for you, the context it sees may contain tokens, and that context may then be recorded by another system. Before connecting any AI tool to an internal system — especially a gateway that holds secrets — ask: will its output be logged, and who can read those logs? Few people asked that question seriously before October 21. After this incident, it has to become a default check when adopting AI tooling.

A one-minute self-check: were you actually in the window?

The question most readers really want answered is a simple one: was I hit? The criteria are concrete, not guesswork. First, timing: only installations of an unpinned LiteLLM via pip between 11:39 and 13:23 UTC on October 21, or containers started from the v1.107.0, latest, main-latest or main-stable tags between 11:46 and 14:20 UTC, fall inside the window. Installs outside those times, versions pinned to 1.106.2 or an earlier explicit release, and enterprise private-registry pulls are outside the officially affected scope. Second, traces: from a clean machine, check the suspect environment's site-packages from outside for cook.pth, cook.py and sitecustomize.py, for OpenSSL staging directories beginning with .pc_e_, and for the lock file /tmp/remote_code_loader.lock and the marker file /tmp/.pc_once_72129118c59f5636c0bb. Finding a startup file confirms exposure, but not finding one does not prove cleanliness — image rebuilds and temporary-directory cleanup can erase traces, a point both LiteLLM and Microsoft stress. Finally, check outbound traffic and cloud records: look for POSTs to the lookalike healthcheck address around install time, and for anomalous GetCallerIdentity calls from the instance role in CloudTrail. If any one of these matches, treat it as exfiltration, not as "probably fine."

One corner is easy to overlook: caches and mirrors. Even after production is moved to 1.106.2, a copy of the 1.107.0 wheel or image layer sitting in an internal pip mirror, a uv cache, a CI build cache or an old image layer can pull the poisoned artifact back into the next build. That is exactly why Microsoft's guidance lists caches, mirrors, lock files and build outputs alongside installed environments. The last item on any self-check should be sweeping those locations too, and removing floating tags such as latest from every deployment configuration in favour of explicit versions plus digests. Only then is the door genuinely closed.

One final, less technical point. LiteLLM's disclosure rhythm deserves credit: on the day of the incident it published the affected versions, the exposure windows, the detection method and the remediation checklist, and days later it openly flagged which early assumptions had since proven inaccurate or incomplete, pointing readers to a canonical record. That "actionable information first, corrections afterwards, nothing hidden" approach is the rarest quality in supply-chain incident response. Compare it with disclosures that say only "we are aware of an issue" and leave users with nothing they can act on. When choosing dependencies, alongside star counts and features, it is worth checking how a project talks when things go wrong — because the next 1.107.0 will come sooner or later, and the only question is whether you will have a checklist ready to execute on day one.