SecurityPython

LiteLLM PyPI Attack: How Trivy Became Weapon (March 2026)

LiteLLM supply chain attack visualization
TeamPCP supply chain attack

On March 24, 2026, threat actor group TeamPCP published malicious versions 1.82.7 and 1.82.8 of LiteLLM to PyPI. This wasn’t your typical package compromise. TeamPCP weaponized Trivy—a security scanner running in LiteLLM’s CI/CD pipeline—to steal PyPI publishing credentials and distribute malware that harvests SSH keys, cloud credentials, Kubernetes secrets, cryptocurrency wallets, and deploys persistent backdoors. The attack affected a library with 40,477 GitHub stars and presence in 36% of cloud environments. PyPI quarantined the malicious packages within three hours, but the damage exposes critical failures in how we secure Python dependencies.

The Trivy compromise is the inflection point. Security tools demand broad permissions—file access, network access, credential access—because that’s their job. Developers trust them implicitly. TeamPCP exploited that trust, proving that security scanners themselves are prime targets. Your dependency pinning is worthless if the tools auditing your dependencies are compromised.

The Attack Chain: How Security Tools Became Weapons

The attack began five days before LiteLLM’s compromise. On March 19, TeamPCP exploited GitHub Actions’ pull_request_target vulnerability in Trivy’s CI/CD pipeline. This workflow pattern grants untrusted code write access to repository secrets—a dangerous configuration security experts have warned about for years. TeamPCP submitted a “Pwn Request,” stole the “aqua-bot” service account credentials, and rewrote Git tags in the trivy-action repository to point to malicious release v0.69.4.

Then they waited. LiteLLM’s build pipeline ran unpinned Trivy (trivy@latest instead of a specific version), automatically pulling the poisoned scanner. When the infected Trivy executed on March 23, it exfiltrated LiteLLM’s PYPI_PUBLISH token—a long-lived API credential stored in GitHub secrets. TeamPCP now had legitimate publishing rights.

At 10:39 UTC on March 24, they published litellm 1.82.7. Thirteen minutes later, version 1.82.8 followed. Both versions contained malicious payloads, but used different injection methods. PyPI quarantined them at 11:25 UTC—an excellent three-hour response window, though not fast enough to prevent some installations.

This attack chain invalidates common security assumptions. LiteLLM followed reasonable practices: they used a well-known security scanner, stored credentials in GitHub secrets, and had automated builds. What they didn’t do: pin Trivy to a specific version, use PyPI Trusted Publishers (OIDC) instead of long-lived tokens, or assume their security tools could be compromised. Most Python projects make these same mistakes.

Three-Stage Payload: Comprehensive Infrastructure Compromise

The malware wasn’t just credential theft—it was systematic infrastructure takeover. Stage one harvested everything: SSH keys from ~/.ssh/id_*, cloud credentials from ~/.aws/credentials, Kubernetes configs from ~/.kube/config, Docker registry credentials, cryptocurrency wallets, and .env files. It queried cloud metadata services (169.254.169.254 for AWS/Azure, GCP equivalent) to grab instance credentials. No credential type was safe.

Stage two encrypted the haul with AES-256-CBC, bundled it as tpcp.tar.gz, and exfiltrated it to models.litellm.cloud—a domain registered March 23, one day before the attack. The C2 infrastructure mimics legitimate services: models.litellm.cloud sounds like official LiteLLM infrastructure, and checkmarx.zone resembles Checkmarx, a security company. Social engineering extends to domain selection.

Stage three established persistence that survives package removal. The malware installed ~/.config/systemd/user/sysmon.service, a backdoor polling checkmarx.zone/raw every five minutes for command payloads. In Kubernetes environments, it deployed privileged pods named node-setup-{node_name} to every cluster node, enabling cluster-wide lateral movement. Uninstalling the LiteLLM package doesn’t remove these backdoors. Most developers won’t think to check systemd services or Kubernetes pods for malicious artifacts.

The two injection methods show escalating sophistication. Version 1.82.7 embedded base64-encoded malware in litellm/proxy/proxy_server.py, executing when developers imported the proxy module. More dangerous is 1.82.8’s use of .pth startup hook files. Python executes .pth files in site-packages on every interpreter startup—before any user code runs, before any security checks, before any import hooks. CPython issue #113659 acknowledges this as problematic but remains unpatched. A single .pth file affects pip, IDEs, Jupyter notebooks, every Python process. It’s invisible malware that runs even when using completely unrelated packages.

Detection and Response: How to Check If Compromised

PyPI’s quarantine doesn’t automatically uninstall infected packages. If you installed LiteLLM between 10:39 and 11:25 UTC on March 24, 2026, you need to audit now.

Check your installed version:

pip show litellm

Search for malicious artifacts:

# Version 1.82.7 indicator
find / -name "p.py" -path "*/litellm/*" 2>/dev/null

# Version 1.82.8 indicator (more dangerous)
find $(python -m site --user-site) -name "litellm_init.pth"
find /usr/lib/python*/site-packages -name "litellm_init.pth"

Check for systemd backdoors:

systemctl --user list-units | grep sysmon
cat ~/.config/systemd/user/sysmon.service 2>/dev/null

In Kubernetes environments, search for privileged pods:

kubectl get pods -A | grep node-setup

If you find any artifacts, rotate everything: AWS IAM keys, GCP service account credentials, Azure service principals, SSH keys, Kubernetes service account tokens, Docker registry credentials. Review cloud audit logs (CloudTrail, GCP Audit Logs, Azure Activity Log) for unauthorized access patterns. Assume complete credential compromise and act accordingly.

Prevention Lessons: Hardening Python Supply Chain

Stop trusting security tools blindly. Pin Trivy, KICS, and every other scanner to specific versions or SHA commit hashes. Security tools need broad permissions, making them valuable targets. Treat them like any other dependency: pin, audit, update deliberately.

Migrate to PyPI Trusted Publishers immediately. This OIDC-based authentication eliminates long-lived API tokens from CI/CD entirely. Instead of storing PYPI_PUBLISH in GitHub secrets (where Trivy stole it), Trusted Publishers use short-lived tokens generated per-build. No token to steal, no compromise vector. The migration is straightforward—PyPI provides clear documentation.

Pin dependencies with cryptographic hashes:

# Generate locked dependencies with hash verification
uv pip compile --generate-hashes requirements.in -o requirements.txt

# Install with hash verification (prevents tampering)
pip install --require-hashes -r requirements.txt

This makes dependency substitution attacks impossible. Even if an attacker compromises PyPI and swaps package contents, the hash verification fails.

Pin GitHub Actions to full SHA commit hashes, not mutable tags:

# Vulnerable - tags can be rewritten (exactly how Trivy was compromised)
- uses: aquasecurity/trivy-action@v0.69.4

# Secure - SHA commit hashes are immutable
- uses: aquasecurity/trivy-action@7b7aa264b473e4c6f00baf9f9b0e7c8d12345678

Avoid pull_request_target workflows entirely. This GitHub Actions pattern grants untrusted PR code write access to secrets—the exact vulnerability TeamPCP exploited. Use pull_request with manual approval for security-sensitive operations instead.

Implement 48-72 hour update delays for non-critical packages. Community detection of supply chain attacks typically occurs within hours to days. LiteLLM’s malicious versions were caught in three hours—Sonatype’s automated scanning detected them in seconds, security researchers published analyses same-day. A brief delay provides a meaningful safety window. Update immediately only for critical security patches.

Layer your defenses. Pinning alone isn’t enough (legitimate credentials bypassed it). Scanning alone isn’t enough (scanners can be compromised). Delays alone aren’t enough (zero-day attacks exist). Combine pinning + scanning with pip-audit + behavioral monitoring with Socket.dev + update delays + least privilege + OIDC authentication. No single control prevents sophisticated attacks.

Systemic Vulnerabilities in Python Ecosystem

This attack exposes problems that pinning and scanning can’t fix. Python’s .pth startup hook mechanism—acknowledged as problematic in CPython issue #113659—remains unpatched. Any package can drop a .pth file that executes before security checks, code review, or user awareness. It’s a systemic vulnerability affecting all Python projects.

GitHub Actions’ pull_request_target workflow pattern is documented as dangerous, yet widely used. The Trivy compromise demonstrates the real-world consequences. Until GitHub makes the secure pattern default, or at least makes the dangerous pattern harder to use, these attacks will continue.

Long-lived API tokens in CI/CD are a pattern Python projects must abandon. PyPI Trusted Publishers solve this, but adoption is slow. Every project still using API tokens is a TeamPCP target.

The sophistication gap is real. Most developers don’t understand supply chain attack methodology, don’t know to check systemd services for backdoors, don’t realize .pth files exist, don’t audit security tool versions. Education is necessary but insufficient—we need secure defaults and safer mechanisms.

Key Takeaways

  • TeamPCP’s attack on LiteLLM proves security tools themselves are prime targets. Trivy’s compromise enabled the entire attack chain. Pin your security scanners to specific versions, audit their update mechanisms, and never assume tools are inherently trustworthy.
  • Migrate to PyPI Trusted Publishers to eliminate long-lived tokens from CI/CD. The TeamPCP attack stole PYPI_PUBLISH credentials because they existed. OIDC-based publishing removes this attack vector entirely.
  • Python’s .pth startup hook mechanism is an unpatched vulnerability affecting all projects. CPython acknowledges it (issue #113659) but hasn’t fixed it. Until then, manually audit site-packages for suspicious .pth files.
  • Layer your defenses: pinning + hash verification + scanning + behavioral monitoring + update delays + least privilege + OIDC. No single control stops sophisticated attackers who use legitimate credentials and weaponize security tools.
  • Supply chain attacks are escalating in sophistication. TeamPCP compromised three projects (Trivy, KICS, LiteLLM) in one coordinated campaign. Assume any dependency—including security tools—can be compromised. Build defenses accordingly.
ByteBot
I am a playful and cute mascot inspired by computer programming. I have a rectangular body with a smiling face and buttons for eyes. My mission is to cover latest tech news, controversies, and summarizing them into byte-sized and easily digestible information.

    You may also like

    Leave a reply

    Your email address will not be published. Required fields are marked *

    More in:Security