TPU 8tx vs Nvidia Confidential Computing: What to Verify Before Committing

If your organization runs regulated or data-sensitive workloads, the honest answer to “can we serve LLM inference confidentially on TPU 8tx?” is: not evaluable from the announcement. Google’s blog post introducing TPU 8tx, its third eighth-generation system, positions the hardware for “secure and confidential inference” aimed at “regulated enterprises and governments,” but the announcement documents no attestation API, no key custody, no memory-encryption scope, and no tenancy model; Cloud TPU security documentation was not reviewed. Every TPU 8tx claim in this article rests on that single vendor post, observed 2026-11-09. Nvidia’s confidential computing path, by contrast, is benchmarked by independent arXiv preprints, but those studies are also author-reported and unreplicated, and at least one of them states outright that it does not audit security and does not measure attestation. That asymmetry defines the decision. One platform has claims without published mechanics; the other has mechanics without a security audit. A compliance review needs the same artifacts from both, and this article lays out what those artifacts are.

What Google claims for TPU 8tx, and what the post does not document

The vendor announcement is worth reading carefully, because what it asserts and what it specifies are different things. Google writes that “TPU 8tx, our third eighth-generation system, is optimized for secure and confidential inference and is also great for small-scale training. It’s built for regulated enterprises and governments running frontier or proprietary models and autonomous agents alongside their most sensitive data.” The security mechanism is described in one sentence: “Built on the same silicon as TPU 8t and paired with 5th Generation AMD EPYC CPU hosts, TPU 8tx builds security directly into the hardware layer, keeping sensitive data and model weights protected and isolated even while in active use.”

“Protected and isolated even while in active use” is the defining promise of confidential computing: data encrypted in use, not just at rest and in transit. But the announcement does not say what encrypts it, what attests to it, who holds the keys, how tenants are separated, or what happens on error. The post also claims the Boardfly network delivers “up to 2x lower latency compared to TPU 8t with a near-zero performance penalty,” again vendor-reported with no independent benchmark in this review.

One limit should be stated once, plainly: Cloud TPU security documentation was not reviewed, so the announcement’s silence is not proof that Google publishes no attestation or key-management documentation. The practical consequence is the same either way, though: until a reviewer can read those artifacts, the confidential label is a label.

The checklist any confidential-inference claim must satisfy

The verification burden does not change with the vendor. Whether the word “confidential” appears in a Google TPU post or an Nvidia CC-mode whitepaper, a compliance review for regulated serving needs documented answers to five questions:

1. Attestation API. Can a client or auditor remotely verify what code and configuration the workload is actually running, before sending data?
2. Key custody. Where do decryption keys live, and what gates their release? The reference pattern in the literature is a key management service that checks the confidential VM against policy via remote attestation.
3. Memory-encryption scope. What is encrypted (GPU HBM, CPU memory, interconnect), and what is explicitly outside the boundary?
4. Tenancy and error handling. How are tenants isolated on shared accelerators, and what is the failure behavior when the boundary or attestation fails?
5. Deployment constraints. Which machine types, regions, TEE types, and boot configurations does the confidential path require, because those constrain both availability and the audit surface.

The rest of this article applies that checklist to both platforms using the vendor announcement and independent studies available as of 2026-11-10.

Attestation and key custody: the pattern to demand

The clearest statement of what “good” looks like comes not from a vendor but from Google’s Confidential Federated Compute research, which describes the attestation-gated key management pattern directly: “The KMS checks the code running on the CVM against the policy via remote attestation, which CVMs support. The KMS itself runs in a CVM; when an outsider verifies correctness of the KMS, they also confirm the privacy budget.”

Two properties in that description are what a reviewer should ask any confidential-inference vendor to demonstrate. First, key release is conditional on verified code and configuration, so a tampered or misconfigured serving stack simply never receives the keys. Second, the KMS itself runs inside the boundary, so the key path is auditable rather than a trusted black box operated by the platform. If a vendor’s documentation cannot point to an attestation API, a policy format, and a key-custody design with these properties, the offering is asking you to trust the operator, which is exactly what confidential computing is supposed to remove from the threat model.

Notably, this is also the item where both platforms’ documentation is thinnest. The TPU 8tx announcement mentions no attestation interface at all, and the independent Nvidia studies decline to measure attestation overhead or verify its behavior.

Nvidia CC mode on the record: H100 under Intel TDX

Nvidia’s path is far better documented operationally. An independent benchmark of confidential GPU inference on H100 under Intel TDX pins down the deployment constraints on Google Cloud: “Google Cloud requires the a3-highgpu-1g machine type and Intel TDX for this confidential GPU path,” with the study’s own instance running one H100 80GB in us-central1-a. That specificity is useful: it tells a platform team exactly what capacity, region, and TEE the confidential path depends on.

The same paper models the reading habit a compliance reviewer needs. Its setup ran with Secure Boot disabled, and the authors say so: “therefore, this paper evaluates the measured performance behavior of the benchmarked confidential instance configuration and does not make a completeness claim about the security configuration.” A benchmark of a confidential instance is not automatically a benchmark of a correctly configured confidential instance. When a vendor or an internal team cites CC performance numbers, the first question is what the security configuration actually was.

On overhead, the two H100 studies disagree, and the disagreement follows the configuration. A study of performance and cost across CPU and GPU TEEs ran an H100 NVL rented from Azure and reports “throughput penalties of 4–8% that diminish as batch and input sizes grow” for “LLM inference on NVIDIA H100 Confidential Compute GPUs,” a direction that favors the high-utilization serving patterns regulated inference typically wants anyway. The TDX benchmark of the Google Cloud path measures 17.7% and 21.1% drops in global token throughput for Mistral-7B and Qwen3-30B-A3B. The same paper reports average TTFT rising 21.8% and 27.8% and advises teams to “reserve 15–25% additional throughput capacity as an initial planning heuristic.” For capacity planning, use the number measured on the configuration you will actually deploy: single digits on the Azure setup, roughly a fifth of throughput plus a latency reserve on the a3-highgpu-1g and TDX path.

The H100’s memory-encryption scope is, at least, documented. The CPU-and-GPU TEE study reports that command buffers, kernels, and data transfers over PCIe between CPU and GPU are “encrypted and authenticated via a bounce buffer,” while “the HBM memory of H100s is not” and “the NVLINK communication is unprotected when combining multiple H100s”; the same paper notes that the B100s “introduce HBM memory and NVLINK encryption.” That is the level of specificity the TPU 8tx announcement lacks, and it matters because it tells you which assets sit inside the cryptographic boundary and which do not.

Blackwell B200 under CC: what is unaffected, what is disabled, what is unmeasured

The newer-generation picture is better on performance and worse on observability. A benchmark of confidential computing on NVIDIA Blackwell B200 GPUs finds GPU compute, energy draw, and usable memory capacity unaffected by CC mode. Do not average that with either H100 result, though: the 4–8% Azure measurement, the 17.7–21.1% Google Cloud measurement, and the B200 findings are different studies on different configurations and hardware generations. The divergence is a warning against quoting any single “CC overhead” number in a capacity plan.

The operational cost shows up elsewhere. The B200 study notes that GPU “event timing are disabled by design under CC, to keep protected memory from leaking through side channels (§3.3), so plan observability around guest CPU-clock profiling, host-side NVTX ranges, and end-to-end throughput.” If your serving stack relies on fine-grained GPU kernel timing for latency debugging or SLO attribution, CC mode removes that instrument. That is a real migration cost for an inference platform team, and it belongs in the evaluation budget alongside the throughput headroom.

The same paper supplies the most important sentence for a compliance reviewer: “It does not audit any security property. Whether the confidential boundary actually holds against a given adversary is out of scope. This is a performance study that takes the guarantee as given and measures its price. Attestation and confidential-VM startup overhead are not measured.” The most rigorous public work on Nvidia CC explicitly refuses to vouch for the security property. That is not a criticism of the researchers; it is the correct epistemic posture, and it is exactly why a folder of CC benchmarks cannot substitute for a security review.

Side channels: why memory encryption is not the whole threat model

TPU 8tx’s headline promise is protection of “sensitive data and model weights… even while in active use.” The side-channel literature is the reason to ask what “protected” excludes. A systematization of knowledge on machine learning with confidential computing warns that “side-channel attacks on TEEs like timing, power, and Electromagnetic analysis could compromise the ML model privacy on some levels, e.g., reconstructing the model architecture and even model parameters.” Model weights are the asset TPU 8tx claims to keep isolated, and the literature says TEE boundaries have historically leaked precisely that class of asset through non-cryptographic channels.

The exposure extends to the data side. The SNPeek study formalizes how “an attacker in the CVM threat model can recover information about sensitive inputs using side channels,” including “personal attributes that are processed by software running in the CVM (e.g., location, webpage visits), not just cryptographic keys.” For healthcare or financial serving, the prompt contents are often the regulated asset, and this result says the CVM boundary alone does not settle their confidentiality.

Nor is this only an academic concern about other people’s platforms. The Confidential Federated Compute paper describes finding, in its own production-intended system, “a set of side-channels… that a hypothetical insider could exploit to circumvent differential privacy (DP) guarantees,” with DP mitigating two of them. Even a sophisticated, attestation-gated confidential platform shipped exploitable side channels. The review question for any vendor, TPU or GPU, is therefore not “is memory encrypted?” but “what side-channel analysis has been done, what was found, and what is the mitigation story?”

Side by side: what the announcement and the studies establish

The table is the argument in miniature. TPU 8tx’s column is mostly empty because the announcement documents none of these items, not because Google has necessarily done nothing. Nvidia’s column is populated, but the most security-relevant cells say “disclaimed” or “unmeasured” rather than “verified.”

Capacity-planning aside: TPU 8i Max and long-context serving

A secondary claim in the same announcement matters independently of the confidentiality question. Google says TPU 8i Max delivers “double the compute, double the SRAM and double the HBM bandwidth of TPU 8i,” and “because TPU 8i Max fits twice the silicon within the existing rack footprint, data centers can deploy significantly more computing power per square foot and per watt.” For teams planning long-context agentic serving, where context length drives memory bandwidth and SRAM pressure, a per-rack doubling of both is the kind of number that reshapes capacity models. Treat it as a vendor projection until independent benchmarks exist, but it is worth tracking, since it changes rack-level economics even for workloads with no confidentiality requirement.

What to do before committing

The practical verdict is asymmetric but convergent. Do not commit to TPU 8tx confidential serving yet: request the attestation API documentation, the key-custody and KMS design, the memory-encryption scope, and the tenancy and error-handling model from Google, and treat the announcement’s security language as a positioning statement until those artifacts arrive. If you need confidential inference in production now, Nvidia CC mode is evaluable today, with known deployment constraints (a3-highgpu-1g plus Intel TDX on Google Cloud), measured overhead on two H100 configurations (4–8% on an Azure-rented H100 NVL, 17.7–21.1% throughput loss on the Google Cloud path, and compute, energy, and memory unaffected on B200), and a real observability cost from disabled GPU event timing. But budget for the security review yourself, because the public studies explicitly do not provide one.

For either platform, I would treat the attestation and key-custody documentation as the gating artifact, ahead of any performance evaluation. A 17.7–21.1% throughput penalty on the Google Cloud H100 configuration is negotiable in capacity planning; a missing answer to “who releases the keys, against what verified measurement” is not. And in both cases, put side-channel exposure on the review agenda explicitly, because the strongest available evidence says encrypted memory alone does not close the threat model.

Limitations

This assessment has hard edges worth stating plainly. The TPU side rests on one vendor blog post, and Cloud TPU security documentation was not reviewed; checking that documentation could change the TPU column of the table quickly. The Nvidia-side numbers come from unreplicated preprints measuring specific configurations (an Azure-rented H100 NVL, one H100 80GB with Secure Boot disabled, a B200, a single zone), take the confidentiality guarantee as given, and do not generalize across generations or providers. And nothing cited here establishes that any platform’s confidential boundary holds or fails against a real adversary. That is precisely the gap a compliance review exists to close, and no announcement closes it for you.