Private Vector Search vs TEEs: Can RAG Retrieval Be Outsourced Safely?

A privacy-sensitive RAG stack has exactly three ways to answer a nearest-neighbor query today: self-host the vector index and own the machine, rent a trusted execution environment and trust the silicon, or outsource to a cryptographic protocol that (in theory) lets a hosted service search your corpus without seeing your queries or your documents. The third option has historically been a joke measured in minutes and tens of gigabytes per query. A preprint that appeared on arXiv this week, Spruce (arXiv:2609.03376), claims to have dragged it into the 0.21, 2.97 second range over corpora up to 5.42 million documents. Those numbers are author-reported, single-paper, and unreplicated. They are also the first evidence that outsourced cryptographic retrieval might be approaching interactive latency, which makes the build-vs-buy question worth re-opening.

What actually leaks when you outsource retrieval?

Outsourcing a vector index exposes two assets, and most teams only think about one of them. The corpus is the obvious one: if your embeddings live on someone else’s hardware, the host can read them, and dense embeddings are reversible enough in practice that “it’s just vectors” is not a privacy argument. The less obvious leak is the query stream. A query log is a record of what your users asked, in what order, and how often. For a legal, medical, or competitive-intelligence workload, the query stream can be more sensitive than the corpus itself.

The three deployment options divide by which of these they protect and from whom:

• Self-hosted (Qdrant, pgvector, your own box): nothing leaves your trust boundary. The host is you. You pay for that in operational surface and in giving up elastic managed infrastructure.
• TEE-based confidential computing: queries and index run inside a hardware enclave, so the cloud operator’s software stack is (in principle) cut out of the trust set. You still trust the silicon vendor, the attestation chain, and the assumption that this year’s side-channel mitigations hold.
• Cryptographic outsourced retrieval: the host holds only compact codes and shares of secret data, and answers queries under a multi-party computation protocol without seeing plaintext on either side. You trust the protocol’s security assumptions and its non-collusion model instead of a machine or a chip.

That last option is the one that has never been practical. The rest of this piece is about whether a new result moves it from “never” to “worth tracking.”

What did private retrieval cost until now?

The honest baseline, quoted from Spruce’s own abstract, is that a naive secure implementation of retrieval at million-document scale costs minutes of latency and roughly 90 GB of communication per query. Ninety gigabytes per query is not a latency problem; it is a billing event. Even the recent optimized systems the paper positions itself against, the actual state of the art rather than a strawman, require 10 to 22 seconds per query.

That 10, 22 second range matters for two reasons. First, it defines the bar Spruce claims to clear, so it is the right denominator for the headline speedups. Second, it explains why “just use a TEE” became the default answer for confidential retrieval. When the cryptographic alternative costs an order of magnitude more than a user will tolerate, hardware enclaves win by default even from teams that dislike trusting Intel or AMD. The privacy decision collapses into a deployment-topology decision: if you won’t accept a TEE, you self-host, and outsourcing retrieval is off the table entirely.

A protocol that closes the latency gap changes that logic. It would let a regulated workload hand retrieval to a hosted service without handing over plaintext queries or the corpus, shifting the privacy question from “whose machine does this run on” to “which protocol do I run.” That is the stakes of the Spruce claim.

What does Spruce claim, in numbers?

Spruce replaces corpus-wide embedding scoring, the expensive part under encryption, with Hamming-distance computation over learned compact binary codes, evaluated inside a two-server multi-party computation protocol. The binary codes act as a privacy-preserving candidate generator: the MPC scan produces a shortlist, and full-precision reranking happens on that shortlist rather than on the whole corpus. Across four corpora containing 383K to 5.42M documents, the authors report preserving original search quality with median candidate sets of only 382 to 1,952 items, per the preprint.

The reported performance figures:

• At 10 Gbps inter-server bandwidth, full scans take 0.21, 2.97 seconds, which the authors describe as 4.8, 6.7× faster than the closest measured prior work.
• On the largest corpus tested, pruning plus the protocol’s dealer component jointly improve sustained throughput by 31.5× at 1 Gbps per link.

Two details in the fine print deserve attention before the headline numbers get quoted around. The 0.21-second best case presumes 10 Gbps between the two MPC servers; at 1 Gbps the system leans on the pruning and dealer optimizations for that reported 31.5× sustained-throughput gain, which is a very different operating point. And quality preservation is reported as median candidate-set sizes. Medians tell you the typical query survives the binary-code pruning; they say nothing about tail queries, which are exactly the ones that matter when the retrieval system is feeding a downstream answer your users will act on.

How do the three options compare?

The comparison below is the practitioner’s version of the decision. One structural caveat up front: the Spruce column rests on measured (if unreplicated) data, while the TEE column’s cost and latency characteristics are drawn from architectural reasoning rather than benchmark numbers in the sources here, so treat those cells as qualitative [unverified].

Two rows in that table carry most of the decision weight. The trust-anchor row is the actual privacy argument: each option moves who you have to trust, and none of them eliminates trust. The verification-status row is the practical one: two of these columns describe things you can deploy this quarter, and one describes a protocol you can currently only read about.

Where does each tradeoff break?

Self-hosting breaks on operational surface, not on privacy. A Qdrant instance wants three open ports: 6333 for the HTTP API and monitoring endpoints, 6334 for gRPC, and 6335 for distributed deployment. Each of those is a thing to firewall, authenticate, patch, and watch. The index-footprint problem at least has a mature answer: Qdrant’s built-in quantization cuts RAM usage by up to 97% with a tunable speed-versus-precision tradeoff, so a multi-million-document index fits in memory that a small team can afford. The failure mode is the team itself. If your threat model includes “our own infrastructure gets subpoenaed or compromised,” self-hosting protects nothing, because the plaintext lives exactly where the attacker or the warrant arrives.

TEEs break on the trust assumption you have to swallow. The enclave model removes the cloud operator’s software from the trust set and replaces it with the CPU vendor, the firmware supply chain, and the attestation service. For many regulated workloads that is a genuinely good trade: auditors understand hardware attestation better than they understand your firewall rules. The failure mode is that the trust is not composable or inspectable. When a new side-channel class appears, your mitigation is a microcode update you did not write and cannot verify. Teams that choose TEEs should be honest that they are choosing a vendor relationship, not a cryptographic guarantee.

Cryptographic retrieval breaks on server compute and bandwidth, historically fatally. The MPC cost multiplier is the reason this column has been academic until now: minutes and ~90 GB per query at the naive end, 10, 22 seconds even in optimized prior systems, per Spruce’s own account of the field. Spruce’s reported numbers attack exactly this axis, moving the scan to cheap Hamming-distance operations over binary codes and pushing work into a dealer phase that the authors credit (jointly with pruning) for a 31.5× sustained-throughput improvement at 1 Gbps on the largest corpus. If those numbers hold, the compute multiplier stops being disqualifying. The new failure mode becomes bandwidth topology: you need two non-colluding servers with 1, 10 Gbps between them, and the interactive-latency claims assume the fast end of that range.

The pattern across all three: each option relocates the risk rather than removing it. Self-hosting moves it to your ops team. TEEs move it to a silicon vendor. MPC moves it to a protocol’s non-collusion assumption and a fat inter-server pipe. The decision is which landlord you’d rather have.

What does a two-server MPC deployment actually require you to trust?

The security claim of a two-server MPC protocol is that neither server alone sees plaintext queries or the corpus; only collusion between the two breaks privacy. In a deployment, that means the two servers must be genuinely independent: different operators, different failure domains, ideally different jurisdictions or business incentives. “Two VMs in the same availability zone” is not a trust model; it is a performance optimization with extra steps.

This is where the protocol-choice framing earns its keep. A self-hosted deployment concentrates all trust in one place you control. A TEE concentrates it in a vendor you cannot audit. A two-server MPC splits it across two parties and gives you a mathematical statement about what each party can learn: nothing, unless they collude. For a regulated workload, that statement is legible to compliance in a way that “the enclave is probably fine” is not. But it only holds if the non-collusion is real, which is an organizational and contractual property, not a property of the code.

There is also a subtlety in what the protocol protects. Spruce’s design, per the preprint, keeps full-precision embeddings available for reranking the candidate set the MPC scan produces. The privacy guarantee applies to what the two servers see during the scan; the reranking path and wherever it executes need their own story. A candidate set of 382, 1,952 items (the reported medians) is small, which helps, but “small” is a performance property, not a privacy one. Anyone evaluating this design should ask precisely which components touch plaintext and under what trust assumption, because the answer is unlikely to be “none.”

What would falsify Spruce’s claims?

The preprint makes concrete, checkable assertions, which is the best thing an unreplicated result can do. The falsifiable core:

• Latency replication. Full scans of 0.21, 2.97 seconds over the four corpora (383K, 5.42M documents) at 10 Gbps inter-server bandwidth. An independent team with the released configuration (if code is released) should be able to reproduce this within a reasonable tolerance.
• Quality preservation. Median candidate sets of 382, 1,952 items that preserve original search quality after full-precision reranking. The test is recall against the plaintext pipeline on the same queries, and specifically the tail behavior the medians hide.
• The throughput claim. The 31.5× sustained-throughput improvement at 1 Gbps from pruning plus the dealer, on the largest corpus. This is the number most sensitive to hardware and network setup, and the one to discount hardest until reproduced.
• The prior-art comparison. The 4.8, 6.7× speedup is measured against systems in the 10, 22 second range. If a faster baseline exists, or if the comparison used a weaker configuration of prior work, the relative claim shrinks even if the absolute numbers hold.

Until at least the first two are independently reproduced, the correct epistemic status of Spruce is “a strong claim from a single group,” and the correct engineering posture is “do not build on this.” None of that makes the paper unimportant. It identifies which numbers the field now has to beat or confirm, and it puts a concrete design (binary codes under two-server MPC, dealer-accelerated, reranking on a small candidate set) on the table for others to attack. That is how a cost curve actually moves.

Which option should you deploy today?

Self-host, or rent a TEE. Those are the only two deployable answers for privacy-sensitive RAG retrieval as of September 2026, and the choice between them is a trust-model question, not a performance one.

Self-host when your threat model is the cloud operator and your team can absorb the operational surface: three open ports per Qdrant instance, quantization tuning (up to 97% RAM reduction if you accept the precision trade), storage, upgrades, and the incident-response burden of owning the machine that holds the plaintext. This is the default for a reason. It is boring, auditable, and fast.

Use a TEE when you need managed infrastructure’s elasticity but cannot hand plaintext to the operator’s software stack, and when your compliance story tolerates trusting the silicon vendor and attestation chain. Accept that you are buying a vendor relationship with a side-channel patch cadence attached.

Track Spruce, don’t deploy it. If the reported numbers survive peer review and independent replication, the calculus changes in a specific way: outsourcing retrieval for regulated workloads becomes economically plausible at multi-million-document scale without trusting a host with plaintext queries or the corpus, and the privacy decision stops being “whose machine” and becomes “which protocol.” The 0.21, 2.97 second scans, if real, put cryptographic retrieval inside the latency envelope that interactive RAG demands, for the first time. That is a large if. The strongest limitation on everything above is that Spruce is a single, author-reported, non-peer-reviewed preprint covering four specific corpora, with quality reported as median candidate sets rather than worst-case tails, interactive latency predicated on 10 Gbps between the MPC servers, no independent replication, and no measured TEE or plaintext self-hosted baseline in the available sources. The non-Spruce rows of the comparison table rest on architectural analogy, not measured data.

The practical move for a team with a privacy-sensitive RAG workload today: deploy the boring option that matches your threat model, and add the Spruce replication question to whatever watchlist you keep for infrastructure that will matter in eighteen months. If the candidate-set approach replicates, the next conversation with your compliance team gets much more interesting, because “we don’t have to trust the host with anything” is a sentence neither self-hosting’s subpoena problem nor the TEE’s microcode problem can currently produce.

Frequently Asked Questions

Does Spruce’s 0.21-second latency claim hold if the two MPC servers are in different cloud regions?

No. The sub-second latency figures assume 10 Gbps inter-server bandwidth, which typically requires servers in the same availability zone or data center. If the servers are in different regions to satisfy jurisdictional non-collusion requirements, latency will likely degrade significantly, potentially reverting to the 10-22 second range of prior optimized systems.

How does Spruce’s binary code approach differ from Qdrant’s built-in quantization for reducing index footprint?

Qdrant’s quantization is a local optimization that reduces RAM usage by up to 97% while keeping the index on a single trusted host. Spruce uses binary codes specifically to enable Hamming-distance computation under multi-party computation, allowing the index to be split across two untrusted servers. Qdrant optimizes for memory efficiency; Spruce optimizes for privacy-preserving computation.

What is the primary operational risk of deploying a two-server MPC setup for RAG retrieval?

The primary risk is the non-collusion assumption. If the two servers are operated by the same entity, share a failure domain, or are subject to the same legal jurisdiction, the privacy guarantee collapses. Unlike TEEs, which rely on hardware attestation, MPC security depends entirely on organizational and contractual independence between the two server operators.

Why is the lack of a measured TEE baseline in the Spruce preprint a significant limitation for practitioners?

Without a direct benchmark, practitioners cannot determine if the 0.21-2.97 second latency is actually faster than a TEE-based solution with similar corpus sizes. TEEs often achieve near-plaintext speeds, so if a TEE can process the same query in under 1 second, the cryptographic overhead of MPC may not justify the added complexity of managing two non-colluding servers.