Can Cryptographic Proofs Show Which Model an LLM API Ran?

Yes, conditionally, and the conditions matter more than the yes. A November 2026 preprint, Sable, describes an argument system whose single-GPU prover can prove that a committed, integer-quantized transformer produced a given token sequence from a given prompt, bit for bit. Every figure in that claim, and in everything else this article reports, is author-reported and unreplicated; of the three mechanisms below, only the statistical audits have been run against live commercial endpoints, and neither the execution proofs nor the attestation deployments have. With that caveat stated plainly, the practical answer for a procurement team is a three-tier one: cryptographic execution proofs are now demonstrated in preprint but carry a reported quality penalty and minutes-scale proving costs; hardware remote attestation proves the machine rather than the math and is the only family the literature reports as low-overhead enough for real-time serving; statistical fingerprinting detects substitution cheaply but its own authors restrict it to triage. Which mechanism fits depends on what your contract actually needs to establish, and on what each technique leaves unverified even when it works.

The gap contracts do not close

When you buy inference from a third-party API, the provider has a standing economic incentive to serve something cheaper than the model on the invoice: a smaller checkpoint, a heavier quantization, a fine-tuned substitute, or a router that sends some fraction of requests to a different backend. Contract terms and periodic audits address this only indirectly. A contract creates liability after you detect a substitution; it does not make detection possible. A one-off audit samples behavior at a moment in time, and a provider routing requests adaptively can pass any fixed probe set.

The auditing literature frames this as a verification problem with three candidate families of answers, each proving something different. The 2025 model-substitution auditing paper puts the core tradeoff in one sentence: zero-knowledge proofs can attest correct inference without revealing inputs or weights, but “face substantial computational costs for large LLMs, making them impractical for real-time APIs today,” while trusted execution environments “provide hardware-level guarantees of integrity and confidentiality with much lower overhead.” That asymmetry, proof strength against overhead, structures every decision below.

Three mechanisms, three different claims

It is worth being precise about what each family establishes, because procurement language often blurs them.

Remote attestation (TEEs). A trusted execution environment attests to a remote party that specific code is running on genuine hardware in an isolated state. This binds the machine and the software stack, not the mathematical output. If the attested code loads the contracted model and your request actually reaches the attested enclave, you get a hardware-rooted argument about what ran. Both conditionals are load-bearing.

Cryptographic execution proofs (zkML, sumcheck/GKR arguments). Here the provider commits to model parameters, a cryptographic commitment that fixes the weights, and then produces a proof that the committed model, executed exactly, generated the served tokens from your prompt. This binds output to a committed artifact. Sable is the latest entry in this line and the reason this article exists now.

Statistical fingerprinting (API-TAIL, KBF). These methods enroll a reference model, probe the API, and test whether the responses are statistically consistent with the reference. They prove nothing in the cryptographic sense; they flag inconsistency cheaply enough to run continuously.

Confusing these three claims is where procurement goes wrong. An attestation marketed as “verifiable inference” proves the box. A fingerprint audit that passes does not prove identity. A bit-exact proof proves the committed model, which, as the next section shows, is not quite the model the provider would otherwise serve.

What Sable actually proves, and what it gives up

Sable’s statement is strong and narrow at once. From the preprint: “We present Sable, an argument system and single-GPU prover for the statement that a committed, integer-quantized transformer produced a given token sequence from a given prompt, bit for bit.” The Sable paper achieves determinism by removing floating point entirely: its provable integer function has no floating-point step, so every rounding, normalization, softmax, and sampling decision is determined by the inputs. That is what makes bit-exactness provable rather than approximate.

The cost shows up in model quality. The same paper reports that this integer function’s “perplexity is 8.3% above BF16’s, against 0.7% for FP8” (Sable). Read that as a procurement statement: the artifact a provider can prove bit-exact is measurably worse than the FP8 artifact a provider would normally serve, by the paper’s own numbers. The authors estimate that wider queries and keys “would recover about a third of the gap at an estimated 30 to 68% more proving time” (Sable), and note that this is an estimate, not a measurement.

So the headline “bit-exact proof” needs one asterisk attached in every procurement document: the proof covers the committed integer-quantized model, not the BF16 or FP8 checkpoint in the marketing page. Whether the 8.3% perplexity penalty Sable reports matters depends on the workload. For a regulated summarization task with acceptance thresholds, it might be disqualifying. For an audit trail on a compliance pipeline, it might be irrelevant. Either way, you are choosing between the provable model and the best model; today you do not get both.

Where the proving time goes

The minutes-per-inference economics of cryptographic proof are the real adoption constraint, and Sable reports its own numbers. Its abstract states that the prover “proves all 64 layers of Qwen2.5-32B in 92.5 s for one sequence of 4,096 tokens, 253 s for four such sequences, and 398 s for one sequence of 16,384 tokens” (Sable). It also states that “A proof takes 41 to 50 times as long as computing the same tokens in our serving engine … or 6.4 cents for a 4,096-token request at public GPU prices” (Sable). The same prover covers Qwen3-32B and the mixture-of-experts Qwen3-30B-A3B, proving 4,096 tokens in 121.8 and 84.7 s (Sable), and the paper reports 40 to 64 times less time per parameter and token than zkLLM. For prior art, Sable’s related-work summary cites earlier sumcheck and GKR systems that “proved convolutional networks [LXZ21] and later language models with up to 13 billion parameters in under fifteen minutes per inference [SLZ24].” Two cautions apply. First, that figure is prior work cited by Sable, not a Sable benchmark. Second, the paper’s own HTML rendering shows the number ambiguously with a duplicated glyph (“1313 billion”); the 13-billion reading is the plausible one, but treat it as approximate. A minute and a half per 4,096-token proof, and about six and a half minutes at 16,384 tokens, is a batch-audit budget, not an interactive API budget, and it is consistent with the independent assessment above that zkML remains impractical for real-time serving.

The deeper reason proofs stay expensive is identified by Artemis, a 2024 paper on commit-and-prove SNARKs for zkML: for some models, “more than 90% of the prover’s time” goes to “commitment consistency checks” rather than ML computation. In plain terms, the prover spends most of its effort proving that the weights it used are the weights it committed to, not on the arithmetic of inference itself. That bottleneck is structural, it follows from the commit-then-prove design, so faster GPUs alone do not remove it. Work like Artemis attacks exactly this cost, which is worth watching, but the evidence in hand says the bottleneck is present in current systems.

The practical consequence: cryptographic proof is viable for workloads that can tolerate minutes of latency per proved inference, post-hoc audits of sampled requests, high-stakes batch decisions, dispute resolution, and not for the chat API on your critical path.

Attestation’s scope and its own trust bill

TEE remote attestation sits at the other end of the overhead spectrum, and the substitution-auditing paper reports it carries “much lower overhead” than zkML; its benchmark of a vLLM endpoint with a TEE enabled measured first-token latency overhead of 9–16% and a 2.88% throughput drop under high concurrency (auditing paper). But attestation has its own prerequisites, and they grow with your fleet.

The TrustMee paper states the problem directly: “However, supporting each new platform enlarges a Verifier’s trusted computing base, since it must include code to verify the attestation evidence from each platform, often performing risky operations such as binary parsing.” Every hardware generation and every vendor stack you accept as attested is more verification code you must trust, and binary parsing is a classic place for exploitable bugs to live. The guarantee “hardware says this code ran” is only as strong as the verifier that checked the evidence.

There is also a scaling cost at deployment. Naive peer-to-peer attestation, where every TEE attests every other, incurs quadratic communication overhead; Careful Whisper proposes a gossip protocol that reduces this “to linear complexity under ideal conditions”, note the qualifier. For a buyer consuming a single provider’s attested endpoints this matters less than for a provider operating a confidential-computing mesh, but it shapes what your provider can credibly offer.

The larger limitation is scope. Attestation proves the machine and the loaded code. It does not prove that the load balancer routed your request into the attested enclave, that the enclave loaded the contracted weights rather than a quantization, unless the attested measurement covers them, or what happens to your data after it leaves the enclave. Mixing and partial substitution, most requests honest, a slice diverted to a cheaper model, are exactly the failure modes the statistical-audit papers were built to catch, which is a hint that attestation alone was not closing them.

Fingerprinting: cheap, sensitive, and explicitly not proof

The third family trades proof strength for cost and coverage. Two recent preprints report what that trade buys.

API-TAIL, a typicality-based audit, “controls false alarms across diverse honest providers and detects low-rate partial substitution with a small query budget,” and its authors report that “[u]nder a common composite calibration, it outperforms the adapted rank- and MMD-based audit baselines.” The partial-substitution point matters: this is designed for the realistic adversary who diverts a fraction of traffic, not one who swaps the whole model overnight.

KBF, which fingerprints a model by its knowledge boundary, reports the strongest detection numbers in the set: “KBF detects all 155 economically relevant substitutions without rejecting any of the 16 same-reference controls” (abstract), and “reaches 95% TPR at a substitution rate as low as 15% in mixed-routing simulations.” Online audits cost approximately $0.67 each after reference enrollment, the paper’s figure for GPT-6 Astra at recorded API prices (KBF). In field audits, KBF’s authors flagged 7 of 28 endpoints across six platforms as statistically inconsistent with their enrolled references, a striking number, though “statistically inconsistent” is doing careful work in that sentence, and the authors do not claim those seven providers were cheating.

They are explicit about the limit: “KBF should be used as a statistical triage tool, not as standalone proof of provider intent or backend identity.” A rejected audit shows inconsistency on the enrolled probe set under the audited interface. Deployment variation, a provider’s legitimate model update, or a changed system prompt can shift statistics without substitution. Fingerprinting tells you where to look; it does not convict.

The comparison a buyer actually needs

Every row has a documented failure surface. There is no mechanism here that a buyer can adopt and stop thinking.

What no mechanism verifies

The checklist of residual risk is identical across all three families, and it belongs in procurement language verbatim:

• Prompt handling. Nothing proves your prompt was not logged, retained, or used for training. That is contractual.
• Routing. Neither attestation nor fingerprinting proves every request took the attested or audited path. KBF’s mixed-routing simulations exist precisely because partial routing is the realistic cheat.
• Downstream data use. Outputs, embeddings, and logs leaving the system are outside every proof and every enclave boundary described here.
• The production model itself. Bit-exact proof binds output to the committed integer artifact, with its reported 8.3% perplexity penalty (Sable), not to the checkpoint the provider advertises.

Treat any vendor claim that one of these mechanisms “verifies the model” as a prompt to ask which row of the table above they mean, and which rows they are not mentioning.

The verdict, with its limits

For a production API on the critical path today, the evidence supports a default of TEE remote attestation plus explicit contract terms covering routing, logging, and data use: attestation is the only mechanism the literature reports as low-overhead enough for real-time serving, and the contract covers what attestation cannot. The auditing paper is candid about where deployment stands: “Adoption remains limited by operational complexity and weak provider incentives, but TEEs are currently the best compromise between verifiability, efficiency, and protection of proprietary models.” Layer statistical fingerprinting, KBF at roughly $0.67 per online audit (KBF), API-TAIL where partial substitution is the concern, as continuous triage that tells you when to escalate. Reserve cryptographic execution proofs for workloads that are high-stakes and latency-tolerant: sampled post-hoc audits, regulated batch decisions, or disputes where a proof that some committed model produced this output settles the question that matters.

I would not yet write a Sable-style proof requirement into a contract as a delivery obligation. The system is a single unreplicated preprint; its overhead and perplexity figures are author-reported; and none of these papers reports a commercial provider deploying any of these three mechanisms. What the preprint changes is the ceiling of what is demonstrable, not the floor of what is deployable. If independent replication confirms even the rough shape of the reported costs, the procurement conversation shifts from “providers cannot prove what ran” to “here is the per-request price of proof”, and that is a negotiation most engineering leads would rather have. Until then, specify in every contract which mechanism proves what, and put the residual list above in writing, because nothing in this literature covers it for you.

Frequently Asked Questions

How long does Sable take to prove a 4,096-token sequence?

Its abstract states that the prover “proves all 64 layers of Qwen2.5-32B in 92.5 s for one sequence of 4,096 tokens, 253 s for four such sequences, and 398 s for one sequence of 16,384 tokens” (Sable).

What is the reported cost of a KBF online audit?

Online audits cost approximately $0.67 each after reference enrollment, the paper’s figure for GPT-6 Astra at recorded API prices (KBF).

What latency overhead does TEE attestation add to inference?

its benchmark of a vLLM endpoint with a TEE enabled measured first-token latency overhead of 9–16% and a 2.88% throughput drop under high concurrency (auditing paper).