Sometimes, yes. A September 2026 preprint reports that forcing a clinical-diagnosis agent to keep gathering evidence raised its error rate from 28.3% to 34.3%; a second paper, accepted to EMNLP 2026 Main, reports curated tool selection lifting ToolBench success from 0.737 to 0.898 on an unchanged agent. Direction depends on evidence handling, not tool count. The clinical-harm and internal-signal findings below come from single, non-peer-reviewed arXiv preprints, and the harm result is from one clinical setting, not general MCP stacks.
What is demonstrated, and what is only asserted
The connector layer is described as pure capability. The July 2026 revision of MCP’s introduction presents an open-source standard for connecting AI applications to external systems, so apps like Claude or ChatGPT can reach data sources such as local files and databases, tools such as search engines and calculators, and specialized prompt workflows. Microsoft’s Learn MCP Server page offers a remote server over streamable HTTP that lets clients like GitHub Copilot pull current official documentation. Both pages describe what a connection makes available. Neither measures what those outputs do to the accuracy of the work downstream.
That distinction organizes everything in this article. Connectivity is an asserted capability; vendor documentation stakes no claim about downstream accuracy. The demonstrated effects come from three preprints posted in September 2026. Two remain unreviewed, since arXiv approves e-prints after moderation but does not peer review them; the ToolBench menu paper is accepted to EMNLP 2026 Main. One shows gathered evidence making an agent worse. One shows an agent-internal signal that predicts whether an action will succeed. One shows curated tool access making an agent substantially better. Together they break the assumption that each new MCP server is a monotonic capability gain, and they relocate the decision from “can we connect this?” to “what does this do to task accuracy?”
Forced evidence-gathering made diagnosis worse
The harm result comes from Safe to Stop? Risk-Constrained Stopping for Sequential Clinical Diagnosis Agents. The setting is sequential clinical diagnosis on a 1,834-episode abdominal-pain benchmark derived from MIMIC records. The comparison that matters holds the agent fixed and varies only the evidence it is required to consume. Diagnosing from the patient’s initial presenting history alone, the agent errs on 28.3% of episodes. Forced through a full evidence workup, error rises to 34.3%. Roughly six points of accuracy disappear because the agent received more information.
The preprint calls this non-monotone: additional evidence does not help uniformly, and at some point in a workup it starts hurting. Its proposed remedy is framing evidence gating as risk-constrained stopping, deciding episode by episode when further gathering no longer pays. Crucially for that framing, the bad state is detectable: a full ranker achieves an exploratory state-error AUROC of 0.853 for flagging error-prone states on the benchmark. The agent’s situation carries a signature that precedes the wrong answer.
Three limits keep this from being a general indictment of tools. Forced continuation is a policy, not the default behavior of most agent stacks, where the model chooses whether to call a tool. The domain is sequential clinical diagnosis, a setting where a workup can pile up redundant and misleading findings in a way a quick web lookup does not. And nothing in the fetched evidence shows this dynamic in a general-purpose MCP stack. The right reading is an existence proof: tool-returned evidence can degrade reasoning in at least one measured setting, so “more inputs” cannot be treated as a default win anywhere it has not been tested.
The counter-case: selection beat accumulation
A second preprint shows the same lever pulled the other way. The Menu Is an Execution Prior studies online agents on ToolBench, a general-purpose tool-use benchmark. The agent stays fixed; what varies is which tools are offered when. A State-Path Tool Menu, curating the visible tool set by state, raises online success from 0.737 to 0.898. That is a 16.1-point absolute gain, achieved “without changing the agent,” and it outperforms retrieval, reranking, generation, and routing baselines.
Read the two results together and the anti-sprawl thesis sharpens into something narrower and more useful. Tools are not inherently harmful: with the same agent and the same benchmark family, better orchestration produced a large gain. Unmanaged evidence accumulation is what hurt, in the clinical setting, under a policy that forbade stopping. The common variable in both papers is evidence handling: which outputs the agent sees, and at what point in its trajectory it sees them.
One caution on comparing the numbers directly. The clinical result is an error rate on sequential diagnosis; the ToolBench result is online task success on general tool-use queries. Different domains, different metrics. What transfers is not the magnitude but the design lesson: the offered set of tools is itself a decision with measurable consequences, not neutral plumbing.
Can the agent tell when a tool call is going wrong?
Gating needs a signal, and two of the preprints supply candidate ones from different directions.
The clinical paper’s ranker, at AUROC 0.853, shows that error-prone states can be flagged from observable episode information. Do Agents Know When They Succeed? goes inside the model. It introduces two complementary methods: Latent Trajectory Dynamics (LTD), which summarizes changes in residual-stream representations across an interaction trajectory, and the Action Representation Probe (ARP), which predicts success from the representations formed at the moment of an action decision. In plain terms, the model’s internal state at the point of choosing a tool carries information about whether that choice will work.
Two practical cautions follow. First, residual-stream methods require white-box access to model internals; an API-only team cannot run ARP or LTD as described, which pushes most production stacks toward observable proxies like the clinical ranker’s inputs. That is an inference about deployment, not a result either paper reports. Second, AUROC 0.853 is a discrimination score: it says the detector ranks risky states well. The clinical paper does report one step past detection into a deployed gate: on its 367-episode evaluation split, the Cros stopping layer yields 16.9% selective error at 78.8% coverage, versus 30.8% error at 100% coverage under native stopping. But the authors bound how far those numbers travel. Cros nominally satisfies the joint criterion in only 6 of 20 development resplits, and because evaluation labels were inspected during earlier development, the paper describes its findings as “exploratory feasibility and audit evidence, not a confirmatory safety certificate.” Treat the gated numbers as evidence that a gate is buildable, not as a confirmatory safety result.
When to fetch, and when to keep reasoning
The evidence condenses into a working rubric. Nothing here is a published standard; it is what the three papers, taken at face value with their caveats, appear to license.
| Decision axis | What the evidence shows | Working rule |
|---|---|---|
| Direction of a tool’s effect | Can be positive (0.737 to 0.898 with curated menus) or negative (28.3% to 34.3% under forced workup) | Measure per task; never assume a monotonic gain |
| Internal-state signal | State-error AUROC 0.853; LTD and ARP predict action success from representations | If you can observe state or confidence, gate on it rather than on call counts |
| Stopping policy | The harm appeared specifically under forced continuation | Let the agent stop, or gate further gathering on detected risk |
| Selection mechanism | Curated state-path menus beat retrieval, reranking, generation, and routing | Constrain the offered set by state instead of appending tools |
| Domain and stakes | Harm shown in high-stakes sequential clinical diagnosis; gains in general ToolBench tasks | The more sequential and consequential the task, the stronger the case for gating |
| Verification status | The ToolBench result is accepted to EMNLP 2026 Main; the clinical and representations results are single non-peer-reviewed preprints | Replicate on your own stack before committing architecture to it |
Applied to a concrete choice, the rubric suggests reasoning alone when the agent’s state indicates it is on track and the task is within the model’s competence, fetching when a specific state reveals a gap a tool demonstrably fills, and gating when the detector or confidence signal flags trouble, since that is precisely where extra evidence did damage in the clinical setting. Where the stakes are low and queries are one-shot, the gating burden is lighter; the clinical result is the strongest argument for gating in long sequential workflows where errors compound.
How to A/B a tool before you wire it in
The operational takeaway is a measurement protocol, not a tool budget. It is a synthesis of the evidence above rather than a procedure any of the preprints prescribes:
- Fix the baseline configuration first. Name the model, prompt, and harness, and freeze an eval set with scored tasks. Percentage deltas are meaningless without a stated baseline, a rule that applies to the preprints’ own numbers as much as to yours.
- Run the agent with the candidate tool absent. Record task-level accuracy, the metric that decides the decision. Retrieval quality or tool latency does not.
- Run the same agent with the tool’s outputs available, on the identical tasks. Where feasible, test both optional use and a forced-use variant, since the clinical harm appeared specifically under forced continuation.
- Segment the results by episode. The clinical finding was that harm concentrated in particular states, and those states were detectable at AUROC 0.853. If a subset of tasks degrades while others improve, the answer may be state-conditional gating rather than removal.
- Decide on the delta. Keep the tool if accuracy holds or improves; if it degrades, gate its outputs by state or drop it. Treat any future tool the same way, including the free ones.
The cost argument is the point. A remote MCP server over streamable HTTP, like Microsoft’s documentation server, is free to attach, which is exactly why tool sprawl compounds: the price shows up nowhere in the connector’s own accounting. An A/B pass per integration converts that hidden cost into a measured one.
Verdict, limits, and what would change the answer
Treat every new MCP integration as a hypothesis, not a free capability. The fetched evidence shows tool-handling choices alone swinging results roughly six points of error in the wrong direction under forced continuation, and 16 points of success in the right direction with curated menus, in both cases without touching the underlying agent. Prefer mechanisms that shape what evidence the agent sees, stopping rules and state-aware tool menus, over raw accumulation, and put the measurement burden before the wiring rather than after.
The limits are real and should temper how far you push this. The harm and internal-signal results each rest on a single non-peer-reviewed preprint; the ToolBench menu result carries EMNLP 2026 Main acceptance but is still one paper on one benchmark. The harm result comes from 1,834 MIMIC-derived abdominal-pain episodes in one clinical domain, under a forced-continuation policy that most stacks do not use, and the 0.853 figure is detector discrimination, with the gated error numbers exploratory rather than confirmatory. No fetched evidence demonstrates non-monotonic evidence harm in a general-purpose MCP stack; the vendor pages, for their part, demonstrate nothing about accuracy at all, in either direction.
What would change the answer: replication of the non-monotone effect outside clinical diagnosis, peer review of the clinical and representations preprints, and vendor-published per-tool accuracy deltas or MCP-integrated benchmarks that measure downstream task performance. If the harm fails to replicate outside forced continuation, gating remains essential mainly for long, high-stakes sequential workflows, and ordinary lookup stacks can relax. If it replicates in general tool use, accuracy labels on tool integrations stop being diligence and start being table stakes.
Frequently Asked Questions
How much did forced evidence-gathering increase the error rate in the clinical diagnosis agent?
Diagnosing from the patient’s initial presenting history alone, the agent errs on 28.3% of episodes. Forced through a full evidence workup, error rises to 34.3%. Roughly six points of accuracy disappear because the agent received more information.
What improvement did curated tool selection provide on the ToolBench benchmark?
A State-Path Tool Menu, curating the visible tool set by state, raises online success from 0.737 to 0.898. That is a 16.1-point absolute gain, achieved “without changing the agent,” and it outperforms retrieval, reranking, generation, and routing baselines.
What is the recommended first step when evaluating a new tool for an agent?
Fix the baseline configuration first. Name the model, prompt, and harness, and freeze an eval set with scored tasks. Percentage deltas are meaningless without a stated baseline, a rule that applies to the preprints’ own numbers as much as to yours.
