When Agents Retrieve Unused MCP Tools: What Tool-Index Examples Change

If your agent’s sessions keep surfacing tools nobody asked for, the instinct to fix it by writing richer tool descriptions now has a measured counterweight. A preprint posted this month (arXiv:2610.30867) tested what happens when you take pooled usage examples and fold them into the tool index as extra entries. The result: retrieval of already-used tools improves, but the tools that never made it into the usage pool get harder to find. On two of the three catalogs the authors studied, folding sharply lowered how often a needed but unused tool reached the top five results, and on StableToolBench pooling all agents’ usage under folding cost every tested agent 12 to 13 held-out Hit@5 points on average.

The practical answer for MCP catalog owners: do not fold pooled usage examples into index entries by default. Score documentation and usage signals separately, prune tools you do not need to expose, or move tool selection to query time entirely. Then, before committing to any of these, measure held-out-tool recall in your own harness, because everything above comes from one author-reported preprint on its own catalogs and retriever, not from Claude Code, Cursor, or any production harness.

What folding actually trades away

The setup matters. Usage examples are valuable signal: past queries that succeeded with a tool tell a retriever what that tool is really for. The question the preprint asks is where that signal should live. “Folding” means adding each usage example to the index as its own entry, so the example itself becomes a retrievable document that routes toward the tool.

The paper’s summary of the result is unusually direct: “Adding each usage example to the index as an extra entry (folding) helps the other tools but, on two of three catalogs, sharply lowers how often a needed unused tool reaches the top five” (arXiv:2610.30867). The mechanism is not hard to picture. Folded examples are concrete, query-shaped text, and query-shaped text matches new queries well. A tool with a rich history of folded examples crowds the top of the ranking. A tool with no usage history, exactly the one a user is asking about for the first time, has only its bare description competing against all that well-matched text.

The held-out measurement makes this concrete. StableToolBench is a catalog of 3,180 real API tools (arXiv:2610.30867), and its held-out set consists of the 172 gold tools in query groups G1-tool and G1-category, whose queries target tools or categories deliberately kept out of the training data. Hit@5 here means whether the correct tool appears anywhere in the top five retrieved results. Two comparisons matter. Against a documentation-only index, folding itself lowered held-out Hit@5 from 0.733 to 0.400 on StableToolBench with Jina, which the paper summarizes as about 30 points lost on StableToolBench and TOUCAN, while labeled Hit@5 rises. Separately, when each agent retrieves with all agents’ pooled usage instead of its own, “pooling under folding lowers every agent’s held-out Hit@5, by 12 to 13 points on average” (arXiv:2610.30867). Drops of this size on held-out tools are the price of better retrieval on the tools that already have history.

Two things this result does not say. It does not say usage examples are useless; the same paper reports folding helps already-used tools. And it does not say descriptions should stay thin. It says the placement of usage signal, inside the same index as tool descriptions, is what creates the crowding-out.

Why catalogs overflow in the first place

The enrich-versus-prune question only exists because forwarding everything stopped being viable. The token math escalates quickly. One MCP server with 26 tools “can consume over 4,600 tokens” of context when all descriptions are forwarded to the model, according to a PwC-authored method paper introducing Tool-to-Agent Retrieval. That is one server. PayPal’s production MCP gateway describes “a tool discoverability barrier where users and agents cannot identify the best tool among 2,000+ indexed tools across 200+ MCP servers” (arXiv:2608.23992).

The reported production cost at that gateway is the starkest number in the cited reports. PayPal’s SCOUT system “reduces MCP tool-token consumption from 140.2k tokens (70.1% of context) to 1.3k tokens (0.8%), a 99% reduction, cutting per-query inference cost” (arXiv:2608.23992) by injecting only the tools relevant to the current step. Those figures are the vendor’s own production report, not an independent audit, but even taken with that caveat they establish the order of magnitude: at gateway scale, tool descriptions were consuming the majority of the context window before retrieval gating.

So there are really two problems stacked on each other. The first is budget: you cannot forward a large catalog. The second is selection: whatever subset you do forward or retrieve can contain the wrong tools, and that is where description quality and index structure come in. The folding result speaks to the second problem. SCOUT speaks mostly to the first.

Three levers, compared

The cited work supports three distinct interventions, and they operate at different layers. Conflating them is the easiest mistake to make here.

ZoE (Z-scores over Evidence) is the preprint’s own alternative, and its design follows directly from the folding failure. It scores documentation and usage separately, so a tool without any usage history keeps its full documentation score instead of being buried by example-rich competitors. The reported comparison: “On all three catalogs, with both embedding models, ZoE finds unused tools more often than a router reserving some of five slots for usage (78.5% against 68.3% of queries on one)” (arXiv:2610.30867). The slot-reserving router, which guarantees some top-five slots go to usage-matched entries, is essentially a softer version of folding, and it still loses on unused-tool recall.

Query-time retrieval takes a different position entirely: do not fix the index, bypass it. In RAG-MCP, “the numerous available tool descriptions (MCP function schemas, usage examples, etc.) are stored in an external memory indexed by their semantics,” and a relevant subset is retrieved per query rather than presenting all tools at once (arXiv:2505.03275). SCOUT applies the same idea in a production gateway. Note the layer difference: these systems change what the model sees per query, not how the index ranks. They solve the token problem directly and the selection problem only as well as their retriever does. If the retriever in front of that external memory folds usage examples into its index, the folding paper’s warning would apply underneath, though no external-memory retriever was tested in the cited work.

Pruning is the row with the weakest direct evidence. Nothing in the cited work measures pruning against enrichment head to head. What the token math supports is narrower: every tool you expose costs context budget when forwarded and costs index surface when retrieved, and 26 tools already exceed 4,600 tokens (arXiv:2511.01854). Removing tools you cannot justify exposing is the one intervention with no retrieval downside to measure.

A same-day diagnostic before you change anything

Because every headline number here comes from one preprint’s catalogs and retriever, the honest recommendation gates on measuring your own harness. The substrates for that already exist.

Held-out-tool Hit@5. Build or borrow a query set whose gold tools were never in your usage pool, then check whether your retriever places them in the top five. HumanMCP provides queries generated to match 2,800 tools across 308 MCP servers (arXiv:2602.23367), and ToolRet offers 7.6k retrieval tasks over a 43k-tool corpus for retrieval measurement beyond MCP specifically (arXiv:2503.01763). The critical discipline is the holdout: reserve tools with no usage history and queries targeting them, mirroring what StableToolBench’s G1 groups do. If you only measure on tools with rich usage history, you will reproduce the “folding helps” half of the finding and miss the half that hurts.

Over-selection, measured separately. Retrieval noise is not the only selection error. Work on over-privileged tool selection reports OPUR rates above 30% for six of eleven evaluated models, with Qwen3-8B at 64.9% (arXiv:2606.20023); LLaMA-3.1-8B sits at 55.9%, while Claude 4.6 Sonnet, GPT-5.2, and GLM-5 remain below 10%. OPUR measures selecting tools with more privilege than the task needs, which is a different failure than retrieving an irrelevant tool, but the same harness logs can feed both diagnostics. If your model grabs powerful tools when narrower ones exist, rewriting index entries will not fix it; that is a model or gating problem.

Run both diagnostics before and after any index change. A 5-point held-out Hit@5 drop in your own harness is worth more than a 12-point drop in someone else’s.

Curation rules for MCP listings

Translating all of this into maintenance practice:

Enrich descriptions, not usage history, inside the index entry. The folding result indicts request-shaped example entries far more than descriptive richness: the preprint’s own text-type control found documentation-shaped entries still lowered held-out Hit@5 by 9.32 points on StableToolBench, just much less than request-shaped text (arXiv:2610.30867). A clear description of what a tool does, its parameters, and its failure modes raises the documentation score that unused tools depend on. Usage examples, if you keep them, belong in a separately scored channel (ZoE-style) or in external memory (RAG-MCP-style), not folded into the same index.

Check what you actually have before writing enrichment plans. A census drawing from a 24,135-server MCP registry found that in a 400-server npm/stdio sample (arXiv:2609.10962), of 194 servers advertising tools, 122 annotate none, and 1,626 of 2,766 tools (58.8%) carry no annotations at all. The census measures optional safety annotations on that npm/stdio slice of the registry; it says nothing about how much usage material a given catalog has to fold. What it does show is that in this sample, basic annotation coverage is the first curation task, ahead of any folding decision.

Prune before you enrich. Enrichment raises the cost of every tool you keep, in writing effort and in index surface. With MCP’s Python and JavaScript SDKs observing over 20 million weekly downloads (arXiv:2602.14878), catalogs keep growing. A tool nobody can justify exposing is cheaper deleted than described. The folding finding adds a subtler reason: a bloated index full of unevenly documented tools is exactly the environment where example-rich tools crowd out the rest.

If you gate at query time, audit the gate. SCOUT and RAG-MCP show the token ceiling is avoidable, but they relocate the selection problem to a retriever you now own. That retriever deserves the same held-out Hit@5 diagnostic as any index change.

What one preprint can and cannot tell you

The limits here deserve plain statement. The folding result is a single author-reported preprint. The effect was measured on its own three catalogs, two embedding models, and its own retriever, with Hit@5 as the proxy for selection quality. No Claude Code, Cursor, or other production harness was tested, and the preprint’s agents are simulated partitions of one usage pool, with no language model in the retrieval loop. Effect sizes could shift substantially with catalog size, embedding model, or annotation coverage, and the 58.8% no-annotation figure (arXiv:2609.10962) suggests many real catalogs look nothing like benchmark ones.

What the preprint can settle is narrower but still useful: in the one measured test, folding pooled usage examples into the index traded unused-tool recall for already-used-tool gains, and scoring the two evidence types separately avoided the trade. That is enough to change a default. It is not enough to skip the in-harness measurement.

My Read, given the evidence as it stands: if you maintain an MCP catalog today, keep index entries descriptive, keep usage signal in a separate scoring channel or external memory, prune what you cannot justify, and hold any enrichment program to a held-out Hit@5 check before and after. If your held-out tools are already surfacing fine and your problem is purely context budget, skip the index surgery and put retrieval in front of the catalog instead. The one thing the current evidence argues against is the easiest-looking fix: pasting usage examples into tool entries and calling the descriptions enriched.

Frequently Asked Questions

What is the practical recommendation for MCP catalog owners regarding folding usage examples?

The practical answer for MCP catalog owners: do not fold pooled usage examples into index entries by default. Score documentation and usage signals separately, prune tools you do not need to expose, or move tool selection to query time entirely. Then, before committing to any of these, measure held-out-tool recall in your own harness, because everything above comes from one author-reported preprint on its own catalogs and retriever, not from Claude Code, Cursor, or any production harness.

What is the token cost of forwarding all tool descriptions from a single MCP server?

One MCP server with 26 tools “can consume over 4,600 tokens” of context when all descriptions are forwarded to the model, according to a PwC-authored method paper introducing Tool-to-Agent Retrieval.