Securing AI Workloads: Why Containers Are AI's Biggest Attack Surface

Containers are the default runtime for AI workloads in 2026—and that convenience comes with compounding risk. Every container vulnerability still applies: escape exploits, misconfigured RBAC, stolen secrets. But AI introduces new attack surfaces on top: model weights worth millions that can be exfiltrated in a single lateral-movement step, prompt injection channels that bypass traditional perimeter defenses, and a model supply chain where malicious weights now circulate at industrial scale.

Why Containers and AI Are a Dangerous Combination

Container security has never been simple. The shared kernel model that makes containers fast also makes isolation imperfect. An attacker who escapes a container escapes into the same kernel space that every other container on the host shares—including any container running proprietary model weights.

AI workloads make this worse in three specific ways.

First, the data density. A single AI training run or inference node may hold terabytes of proprietary model weights, training datasets, and fine-tuning checkpoints. Traditional applications leak credentials or configuration data. AI containers leak assets that cost hundreds of millions of dollars to create and represent core competitive advantage.

Second, the traffic patterns. AI pipelines generate far more east-west traffic than conventional applications. Data moves between preprocessing containers, training nodes, feature stores, inference endpoints, and evaluation harnesses—often across pod boundaries. If an attacker compromises one training container, traditional network controls may allow them to traverse to the entire pipeline and exfiltrate everything, as Tigera’s analysis of AI workload network behavior demonstrates.¹

Third, the agent dynamic. Modern AI agents—models given tool access and the ability to execute code—can be weaponized through the application layer rather than the infrastructure layer. Attackers don’t need to exploit a kernel vulnerability if they can craft a prompt that causes the agent to exfiltrate data on their behalf.

The NVIDIA Container Toolkit Problem

The most concrete evidence that AI infrastructure has a serious container security problem sits in the NVIDIA Container Toolkit CVE record.

In September 2024, Wiz Research disclosed CVE-2024-0132, a container escape vulnerability in the NVIDIA Container Toolkit that allowed a malicious container image—when executed by the vulnerable toolkit—to escape its isolation and gain full root access to the host machine. The mechanism was a subtle misconfiguration in how the toolkit processes OCI hooks. Critically, Wiz found the vulnerability exploitable with a three-line Dockerfile.³

The blast radius: Wiz estimated the vulnerability affected over 35% of cloud environments at time of disclosure—every organization running GPU-accelerated containers with the standard NVIDIA toolkit.

In July 2025, a second critical flaw followed. CVE-2025-23266 (dubbed “NVIDIAScape” by Wiz) carried a CVSS score of 9.0 and affected NVIDIA Container Toolkit versions up to and including v1.17.7. An attacker with permission to deploy a container in a Kubernetes cluster—a relatively low bar in environments with permissive RBAC—could exploit it to escape to the host and access sensitive data from any other tenant on the same hardware.⁴

For AI workloads specifically, this matters in a way it does not for typical web applications. The NVIDIA Container Toolkit is required to give containers access to GPU hardware—it is not optional for AI inference. Organizations cannot simply remove it. They can only patch and monitor.

Model Supply Chain: Attacks Before the Container Starts

Container runtime security addresses threats after deployment. The AI model supply chain represents a different class of threat: attacks that compromise model weights before the container ever starts, embedding malicious behavior that persists through every deployment.

The scale of this problem in 2024-2025 exceeded most practitioners’ expectations.

JFrog’s 2025 Software Supply Chain Report documented over one million new models uploaded to Hugging Face in 2024 alone—along with a 6.5-fold increase in malicious models year-over-year.⁵ Protect AI’s scanning of 4.47 million model versions found 352,000 unsafe or suspicious issues across 51,700 models.⁶

The primary attack vector is Python serialization. The .pkl (Pickle) format—ubiquitous for storing PyTorch model weights—executes arbitrary code on deserialization. A model file that exploits this is indistinguishable from a legitimate model file until it runs. The standard defense tool, Picklescan, was bypassed in early 2025 when ReversingLabs discovered a technique they named “nullifAI,” which evaded detection while preserving malicious payloads.⁷

The Hugging Face platform itself became a direct attack vector in June 2024. The company disclosed unauthorized access to its Spaces platform, where authentication secrets—API keys and tokens used by developers and organizations to access APIs and services—were exfiltrated. Hugging Face revoked compromised tokens, brought in external forensics, and notified law enforcement.⁸ The incident underscored a structural risk: AI teams treat model repositories like package registries, pulling weights at build time from external sources with limited verification.

Sidecar Injection: A Stealthy Persistence Mechanism

The Kubernetes sidecar pattern—running a helper container alongside the primary application container in the same pod—is standard practice for logging, service mesh proxies, and secrets injection. For attackers who have already compromised a cluster, it is also a persistence and exfiltration mechanism that many detection tools miss.

Microsoft’s Threat Matrix for Kubernetes documents sidecar injection (MS-TA9011) as a distinct technique: after compromising a cluster, an attacker modifies pod specifications to inject a malicious sidecar container into a running workload. Because the sidecar runs in the same network namespace as the primary container—including an AI inference server—it can intercept requests and responses, log prompts and completions, and exfiltrate model interactions without triggering alerts on the primary container.⁹

For AI workloads, this creates a specific threat: prompt and response interception at scale. An attacker with sidecar access to an inference pod has access to every query sent to a model and every response returned—potentially including users’ private data embedded in prompts, proprietary system prompt configurations, and fine-tuning-revealing response patterns.

The detection problem is compounded by AI agent behavior. As ARMO’s threat detection analysis notes, “the signals that indicate a compromise in a traditional container are indistinguishable from an AI agent doing its job”—agents legitimately perform network requests, spawn processes, and access filesystems as part of normal operation.¹⁰

The Threat Surface Breakdown

Defensive Architecture for AI Containers

Effective defense operates in three layers: before the container runs, at admission, and at runtime.

Pre-deployment: Treat model weights like code.

Sign model artifacts using Cosign (part of the Sigstore project) and generate Software Bills of Materials (SBOMs) and Model Bills of Materials (MBOMs) that capture weight provenance, training data lineage, and configuration files. Enforce signature verification as a prerequisite for deployment.¹¹ Scan all model files with tools that detect malicious Pickle payloads before they enter the container image. Maintain a private model registry rather than pulling from public sources at runtime.

# Sign a model artifact with Cosign
cosign sign --key cosign.key oci://registry.example.com/models/llama-fine-tuned:v1.2

# Verify signature before use
cosign verify --key cosign.pub oci://registry.example.com/models/llama-fine-tuned:v1.2

Admission control: Enforce at deployment time.

OPA Gatekeeper and Kyverno both serve as Kubernetes admission controllers that reject non-compliant workload specifications before they reach the scheduler. Policies relevant to AI workloads include: requiring read-only root filesystems for inference containers, blocking containers that request excessive GPU access without corresponding resource quotas, enforcing pod security standards that prevent privilege escalation, and requiring signed images from approved registries.¹²

# Kyverno policy: require signed images for AI workloads
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-signed-ai-images
spec:
  validationFailureAction: Enforce
  rules:
    - name: check-image-signature
      match:
        resources:
          kinds: [Pod]
          namespaceSelector:
            matchLabels:
              workload-type: ai-inference
      verifyImages:
        - imageReferences: ["registry.example.com/models/*"]
          attestors:
            - entries:
                - keys:
                    publicKeys: |-
                      -----BEGIN PUBLIC KEY-----
                      <your-cosign-public-key>
                      -----END PUBLIC KEY-----

Runtime: Detect anomalous behavior.

Falco—the CNCF runtime security tool—monitors kernel syscalls via eBPF and triggers alerts on unexpected behavior patterns. For AI inference pods, meaningful rules include alerting on outbound network connections to non-approved destinations (an inference container should not be initiating connections to arbitrary external hosts), detecting unexpected process spawning inside inference containers, and flagging filesystem writes to model weight directories.¹³

For highest-risk workloads—AI agents given shell access or code execution capabilities—Google’s Agent Sandbox initiative (a Kubernetes SIG Apps subproject) provides hardware-enforced isolation built on gVisor with optional Kata Containers support. The project exists specifically because traditional container isolation is insufficient for AI agents executing untrusted or AI-generated code.¹⁴

Network Controls Are Not Optional

The east-west traffic explosion in AI pipelines makes network policy enforcement—not merely network visibility—critical. Without explicit Kubernetes NetworkPolicy objects restricting pod-to-pod communication, a compromised preprocessing container can reach inference pods, model registries, training checkpoints, and any other resource in the cluster.

Apply default-deny network policies at the namespace level for AI workloads, then add explicit allow rules for each required communication path. A fine-tuning job that needs to read from a data lake and write to a model registry needs exactly two outbound rules—not open access to the cluster.¹⁵

Frequently Asked Questions

Q: Do these container security risks apply to managed AI services like SageMaker or Vertex AI, or only self-hosted Kubernetes? A: Managed services abstract much of the container runtime layer, reducing exposure to vulnerabilities like CVE-2025-23266. However, supply chain risks—malicious model weights and compromised dependencies—apply regardless of where inference runs, since the attack occurs before deployment. Managed services also don’t protect against prompt injection or misconfigurations in application-layer access controls.

Q: Is the SafeTensors format safe enough to load model weights from untrusted sources? A: SafeTensors eliminates the code-execution-on-deserialization risk specific to Pickle files, but it doesn’t protect against adversarial model weights—backdoored or fine-tuned weights that behave maliciously at inference time. For untrusted sources, scanning, provenance verification, and sandboxed evaluation are still required.

Q: How do I detect if a sidecar injection attack has already occurred in my cluster? A: Audit pod specifications for sidecars that were not defined in your deployment manifests—any container in a pod that doesn’t match an approved template is suspicious. Falco’s default ruleset includes detection for modifications to running pod configurations. Enable Kubernetes audit logs and alert on patch and update operations on pods in production namespaces.

Q: What’s the minimum viable security baseline for a team deploying AI inference containers for the first time? A: At minimum: pin all base images and model dependencies to specific digests (not tags), enforce non-root user execution in inference containers, apply default-deny network policies to AI namespaces, and implement at least image scanning in the CI/CD pipeline before any container reaches production. These four controls block the most common attack paths without requiring significant operational overhead.

Q: Can prompt injection actually lead to container escape, or does it stay at the application layer? A: Directly, prompt injection stays at the application layer—it manipulates what the model generates, not the container runtime. The risk escalates when an AI agent has tool access that reaches the infrastructure layer: an agent given shell execution capability that receives a malicious prompt can be directed to execute commands that traverse into the host. Container isolation is the last line of defense, not the first.