On March 24, 2026, the TeamPCP threat group published two malicious versions of LiteLLM to PyPI — not by compromising LiteLLM’s application code, but by poisoning the Trivy vulnerability scanner that ran inside its CI pipeline five days earlier. The attack stole a long-lived PyPI publish token from the GitHub Actions runner environment and used it to ship credential-harvesting packages to a library present in approximately 36% of cloud environments monitored by Wiz1.
How the Chain of Trust Broke: From Trivy to PyPI Publish Token
On March 19, 2026, TeamPCP force-pushed malicious commits onto 76 of 77 version tags in aquasecurity/trivy-action and all 7 tags in aquasecurity/setup-trivy2. Because Git tags are mutable references, any workflow pinning to aquasecurity/trivy-action@v0.28.0 or a similar tag continued pulling what appeared to be the trusted security scanner — while running attacker-controlled code alongside the legitimate scan output.
The injected code harvested every secret visible in the GitHub Actions runner environment and exfiltrated them silently2. For LiteLLM, one of those secrets was PYPI_PUBLISH — a long-lived API token stored as a repository secret, scoped to publish on behalf of the litellm package.
Five days after the Trivy compromise, on March 24, 2026 at approximately 10
UTC, attackers used the stolen token to publish litellm 1.82.7. Version 1.82.8 followed thirteen minutes later3.Inside the Payload: Two Delivery Mechanisms and What Made .pth Files Particularly Dangerous
The two releases represent an escalating deployment strategy, not redundancy.
Version 1.82.7 injected a base64-encoded payload directly into litellm/proxy/proxy_server.py at line 128, executing on import4. Any process that imported LiteLLM would trigger the malware — a running proxy server, a test suite, a startup script. For a package downloaded approximately 3.4 million times per day1, that is a significant surface, but it requires the package to actually be imported.
Version 1.82.8 removed that constraint. It planted a litellm_init.pth file inside site-packages45. Python’s .pth mechanism runs code listed in these files on every interpreter startup — before any application code executes, before any explicit import occurs, before pip itself resolves dependencies. A developer running pip install in an environment containing the malicious package would have triggered the payload without ever importing LiteLLM. So would an IDE language server spinning up a background interpreter.
Blast Radius: What Data Was Exfiltrated and to Where
The three-stage payload4 was built to extract maximum credential value from AI development environments:
Stage 1 — Harvest: SSH private keys, AWS IAM credentials, GCP Application Default Credentials, Azure tokens, Kubernetes configs (~/.kube/config), Docker configs, .env files, and cryptocurrency wallets.
Stage 2 — Exfiltrate: The harvested bundle was encrypted using an AES-256 session key wrapped with RSA-4096, then posted to models.litellm[.]cloud as tpcp.tar.gz.
Stage 3 — Persist: A systemd user unit (~/.config/systemd/user/sysmon.service) was written to poll checkmarx[.]zone/raw every 50 minutes for follow-on binaries. On Kubernetes environments, the malware deployed privileged node-setup-* pods across clusters.
The exfiltration and C2 domains reflect a deliberate TeamPCP pattern: project-themed names that blend into network logs without immediately triggering suspicion4.
The Persistence Layer: Systemd Backdoors and Kubernetes Lateral Movement
By writing to ~/.config/systemd/user/ rather than system-level /etc/systemd/system/, the malware operated entirely within user space — no root required, surviving reboots without elevated privileges and evading detection rules tuned toward system-level service creation4.
The 50-minute poll interval is deliberate. It avoids the signature of high-frequency beaconing while still delivering follow-on payloads within an hour of an operator decision.
The kill switch embedded in the payload provides additional operational context: if the URL returned by checkmarx[.]zone/raw contains the string youtube.com, the script aborts4. This is a standard abort mechanism — when a campaign is burned, the operator redirects the C2 response to a benign URL, neutralizing all active implants without reaching individual infected hosts.
The Three Controls That Would Have Stopped This Attack
Each link in the attack chain maps to an absent control. None require custom tooling.
1. SHA-Pinned GitHub Actions
LiteLLM’s workflow referenced the Trivy action by a mutable tag. SHA-pinned references are not mutable: a force-push changes the tag but cannot change the commit SHA a workflow resolves to.
# Vulnerable — tag is mutable- uses: aquasecurity/trivy-action@v0.28.0
# Safe — SHA is immutable- uses: aquasecurity/trivy-action@<full-40-char-commit-sha>Tools such as dependabot with update-config: github-actions or Renovate with pinDigests: true can automate pin maintenance across a repository.
2. PyPI Trusted Publishers (OIDC) Instead of Stored Tokens
This control directly closes the vector TeamPCP exploited. The PYPI_PUBLISH token was a long-lived secret stored in GitHub Actions repository secrets, accessible to any action running in that environment.
PyPI Trusted Publishers exchange a short-lived, job-scoped GitHub OIDC token for a time-limited PyPI upload token6. The OIDC token is cryptographically bound to a specific repository and workflow path. A compromised action cannot reuse the token outside that job’s lifetime.
# With OIDC Trusted Publishers, no stored PYPI_PUBLISH secret exists to steal- name: Publish to PyPI uses: pypa/gh-action-pypi-publish@release/v1 with: attestations: true3. Environment Isolation Between Scanner Jobs and Publish Jobs
Even without OIDC, structuring workflows so the scanner job never has access to publish credentials limits the damage a compromised action can do. GitHub Actions jobs run in isolated environments; secrets can be restricted to specific jobs via GitHub Environments.
jobs: scan: runs-on: ubuntu-latest # PYPI_PUBLISH is not present here steps: - uses: aquasecurity/trivy-action@<commit-sha>
publish: needs: [scan] runs-on: ubuntu-latest environment: pypi-production # Secrets scoped to this environment only steps: - uses: pypa/gh-action-pypi-publish@release/v1Indicators of Compromise and Immediate Remediation Steps
Affected versions: litellm 1.82.7 and 1.82.8 only. LiteLLM has confirmed versions 1.78.0–1.82.6 are clean7. The team released v1.83.0 on March 30, 2026 via a rebuilt CI/CD pipeline; pinning to v1.82.6 or below, or upgrading to v1.83.0 or above, is the recommended safe path7.
If either malicious version may have been installed:
- Run
pip show litellm | grep Version. If the output is1.82.7or1.82.8, treat the environment as compromised. - Check for
~/.config/systemd/user/sysmon.service. If present, disable and remove it, and audit its execution history. - Check all Kubernetes namespaces for pods matching
node-setup-*. Privileged pods of this pattern should be treated as implants and removed. - Check for
litellm_init.pthin every affected Pythonsite-packagesdirectory (python -c "import site; print(site.getsitepackages())"lists them). Remove it manually — upgrading LiteLLM does not remove.pthfiles left by the malicious installer. - Rotate all credentials accessible to the affected Python process: SSH keys, AWS IAM keys, GCP service account keys, Azure tokens, and any secrets in
.envfiles.
Network indicators (as of 2026-04-21): outbound connections to models.litellm[.]cloud and checkmarx[.]zone from application hosts or CI runners.
What This Means for Every AI Python Stack: LiteLLM Is Not the Last Target
LiteLLM was one target in a broader March 2026 campaign. Between March 19 and March 27, TeamPCP also compromised Checkmarx KICS (March 23), Telnyx PyPI packages 4.87.1–4.87.2 (March 27), more than 45 npm packages, and OpenVSX extensions — each wave using a project-themed C2 domain to blend into normal traffic4.
The targeting logic is consistent: security and developer tooling occupies a privileged position in CI pipelines. A vulnerability scanner, an infrastructure-as-code auditor, a communications API — all run with broad secret access in CI, the most permissive environment most organizations operate. Compromising them is higher-yield than targeting application code directly.
For AI-specific stacks, the risk surface is elevated further. LiteLLM sits in the critical path of a growing number of model-routing and agentic architectures. Environments that run it typically also hold API keys for major model providers — credentials that carry significant financial exposure and do not always fit neatly into standard IAM rotation workflows.
The three controls above — SHA-pinned actions, OIDC token scoping, and job-level secret isolation — are baseline CI properties, not remediation steps specific to this incident. The cost of retrofitting them is measured in hours. The cost of skipping them was demonstrated in March 2026.
FAQ
Q: I use LiteLLM as a transitive dependency, not a direct install. Am I affected?
Yes, if your dependency tree resolved to 1.82.7 or 1.82.8 during the exposure window. The .pth file technique used in 1.82.8 fires on any Python interpreter startup in the affected environment, with no import of LiteLLM required45. Check site-packages for litellm_init.pth in every Python environment that could have pulled either version — including CI runner base images.
Q: Does upgrading to v1.83.0 automatically remove the .pth file left by 1.82.8?
No. Upgrading litellm does not remove artifacts written by the malicious installer to site-packages. Delete litellm_init.pth manually from every affected environment; python -c "import site; print(site.getsitepackages())" lists the directories to check.
Q: TeamPCP used project-themed C2 domains. Does that mean standard DNS blocklists won’t catch this?
It means they won’t catch it immediately. Newly registered domains with project-adjacent names (models.litellm[.]cloud, checkmarx[.]zone) are unlikely to appear in threat intelligence feeds at the moment of a campaign launch. Egress filtering by allowlist — blocking all outbound connections except those explicitly approved — is more reliable than blocklist-based detection for this class of attack. CI runners in particular should have tightly scoped egress policies.
Footnotes
-
Wiz, “LiteLLM TeamPCP Supply Chain Attack: Malicious PyPI Packages,” accessed 2026-04-21. https://www.wiz.io/blog/threes-a-crowd-teampcp-trojanizes-litellm-in-continuation-of-campaign ↩ ↩2 ↩3
-
Microsoft Security Blog, “Guidance for detecting, investigating, and defending against the Trivy supply chain compromise,” March 24, 2026. https://www.microsoft.com/en-us/security/blog/2026/03/24/detecting-investigating-defending-against-trivy-supply-chain-compromise/ ↩ ↩2
-
Snyk, “How a Poisoned Security Scanner Became the Key to Backdooring LiteLLM,” accessed 2026-04-21. https://snyk.io/blog/poisoned-security-scanner-backdooring-litellm/ ↩
-
Datadog Security Labs, “LiteLLM and Telnyx compromised on PyPI: Tracing the TeamPCP supply chain campaign,” accessed 2026-04-21. https://securitylabs.datadoghq.com/articles/litellm-compromised-pypi-teampcp-supply-chain-campaign/ ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8
-
GitHub issue #24512, BerriAI/litellm, “CRITICAL: Malicious litellm_init.pth in litellm 1.82.8 — credential stealer,” accessed 2026-04-21. https://github.com/BerriAI/litellm/issues/24512 ↩ ↩2
-
GitHub Docs, “Configuring OpenID Connect in PyPI,” accessed 2026-04-21. https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-pypi ↩
-
LiteLLM, “Security Update: Suspected Supply Chain Incident,” accessed 2026-04-21. https://docs.litellm.ai/blog/security-update-march-2026 ↩ ↩2 ↩3