When LLM Agents Forecast Disease Outbreaks, Who Validates the Forecast?

If your department is deciding whether to cite an LLM-generated epidemic forecast in a situational report this season, the short answer is: not yet, and not because the underlying claim is weak. A preprint from a single team at Google Research, Harvard, and UMass reports that its LLM-guided tree search system, ERA, produced forecasting ensembles competitive with CDC hub ensembles across three respiratory pathogens in a prospective, leak-proof evaluation (arXiv:2605.16238). That claim is unreplicated, self-reported, and published on arXiv by the team that built the system. But the more practical problem is structural: a forecast can be generated months before it can be scored, which means the burden of early warning shifts from the statistical teams that publish methods to whoever in your department must decide whether to trust an unaudited number.

That shift is the real subject of this paper. The question the title asks, who validates the forecast, has a concrete answer in the paper’s own design and a much less concrete one in operational practice.

What the paper actually claims, in its own words

ERA is not a model. It is a system that writes models. The authors describe the mechanism directly:

“Given a natural language task specification, access to historical surveillance data, an evaluation harness with a defined objective function, and a computational sandbox with access to relevant scientific libraries, ERA uses an LLM-guided tree search to explore the space of possible modeling approaches autonomously” (arXiv:2605.16238).

The headline result is narrower than it first sounds. The authors are explicit that no single generated model wins:

“The key finding is not that any single ERA-generated model outperforms all alternatives, but rather that the system can rapidly produce a large, methodologically diverse pool of competitive models whose ensemble matches or exceeds the gold-standard CDC hub ensembles—themselves aggregations of forecasts from dozens of expert teams” (arXiv:2605.16238).

Two things follow for a reader deciding whether to cite this. First, the claim is ensemble-level parity, not a better forecaster. Second, it is one team’s summary of its own evaluation. The authors also state the prospective framing plainly:

“This work demonstrates that an LLM-guided tree search system can generate epidemiological forecasting models that compete with the state of the art across three respiratory pathogens in a fully prospective, leak-proof evaluation setting” (arXiv:2605.16238).

“Fully prospective, leak-proof” is doing heavy lifting here, and it matters that the claim is the authors’ own characterization. The task itself is concrete: weekly forecasts of hospital admissions for influenza, COVID-19, and RSV across 52 US jurisdictions, at horizons of zero to three weeks ahead, each issued as a predictive distribution over 23 quantiles, with models deployed prospectively starting in November 2025 (arXiv:2605.16238). The evaluation uses variants of the Weighted Interval Score, which the paper describes as a proper scoring rule. This article treats the ensemble-parity claim as the authors’ self-reported summary rather than independently checked numbers. Every quantitative claim in this article should still be treated as unreplicated, single-team evidence.

Inside the search: how ERA generates, scores, and selects models

The mechanism is worth understanding because it determines what an auditor can actually inspect. At each node of the search tree, the LLM reads the previous model’s code, modifies it, and the modified model is executed and scored on a validation period. The score steers which branches get explored next. To guard against the search overfitting to that validation window, the system scores every candidate on a second period:

“To guard against overfitting to the validation period, all candidate nodes are additionally evaluated on a separate retrospective test set” (arXiv:2605.16238),

with final models selected on combined performance across both periods.

This two-period guard is a real design choice, and it addresses the most obvious failure of automated model search: hill-climbing on one stretch of history until the model memorizes it. But it also marks the boundary between retrospective skill and prospective prediction. Both the validation period and the retrospective test set are history. The prospective component of the evaluation, weekly submissions made before outcomes are known, is what separates a forecast from a backtest, and it is the part a department should insist on seeing as a timestamped record, not a summary.

The paper’s strongest operational affordance is that the outputs are not black boxes:

“every model in this pool is a complete, executable Python program whose forecasting logic and timestamped outputs can be read, audited, and modified by any domain scientist” (arXiv:2605.16238).

Model source code and weekly forecast submissions are public under Apache 2.0 at the project’s GitHub repository, a contrast the authors draw explicitly with black-box AutoML. Note the asymmetry, though: auditability is an affordance, not a validation. Someone with epidemiological judgment still has to read the generated code, and LLM-generated programs can be plausible, executable, and wrong in ways that only surface at a turning point.

The authors concede two limitations that matter more than the headline. The first is a structural bias shared with the rest of the field:

“models tend to underestimate the rate of increase as the epidemic is rising and the rate of decrease as it falls… Forecasts generated by ERA are no exception” (arXiv:2605.16238).

Underestimating the rate of rise is precisely the failure that costs a health department the most, because the weeks of exponential growth are when a forecast either triggers action or doesn’t. The second limitation is about the system’s own internals:

“Because we evaluated only one implementation of the in-loop judge, these results do not establish whether alternative judge designs or calibration strategies would produce more consistent improvements” (arXiv:2605.16238).

In other words, the component that steers the search was tested in a single configuration. A different judge might produce a different pool of models and a different ensemble.

Reading the baseline honestly: the yardstick is contested

“Matches or exceeds CDC hub ensembles” sounds like the strongest possible endorsement, because hub ensembles aggregate forecasts from dozens of expert teams. The paper also reports that earlier retrospective runs of the search produced models on par with or better than the official COVIDHub ensemble (arXiv:2605.16238). That retrospective history should not be conflated with the prospective result; they answer different questions.

The scoring rule deserves its own scrutiny. Johannes Bracher’s extended note on the FluSight competition metric makes a point every forecast consumer should know:

“To evaluate the participants’ forecasts a multibin logarithmic score has been created, which is a non-standard variant of the established logarithmic score. Unlike the original log score, the multibin version is not proper and may thus encourage dishonest forecasting” (Bracher, arXiv:1910.07084).

A proper scoring rule is one a forecaster maximizes by reporting its honest belief. An improper one can reward strategic distortion. Bracher’s finding is specific about the mechanism:

“We showed that the multibin log score used in the CDC FluSight competitions incentivizes hedging, i.e. tuning forecast distributions in a specific way before submission. While we strongly doubt that participants have consciously tried to game the score, it is possible that this happens unintentionally” (arXiv:1910.07084).

His recommendation is direct: “we therefore advocate the use of proper scoring rules to evaluate epidemic forecasts, e.g. the regular log score” (arXiv:1910.07084).

This history does not indict the ERA result. The paper’s stated metric is a variant of the Weighted Interval Score, which the paper describes as proper, not the multibin log score Bracher showed to be improper (arXiv:2605.16238). What his note still settles is that the evaluation rule is a choice with consequences, and that this field’s record includes rules that rewarded distortion. It sharpens the article’s central question: “who validates the forecast” cannot be answered by pointing at a rank on someone else’s chosen score. Validation has to happen under a scoring rule chosen for its statistical guarantees, by someone other than the team that built the forecaster.

There is also a methodological point about how to compare probabilistic forecasts once you have a proper rule. Work on Gaussian-process forecasting for intermittent series argues against rank-based comparison:

“We argue that models should be compared considering the mean of the quantile losses, and not a rank-based statistic. The distribution of the quantile loss is strongly asymmetric: most forecasts have a small quantile loss; only a few forecasts incur a large penalty. The mean is sensitive to such rare and large losses, unlike rank statistics” (arXiv:2502.19086).

The same authors note that “the correct predictive distribution is guaranteed to minimize the expected value of the scoring rule (Gneiting and Raftery, 2007), but there are no guarantees for rank statistics,” and they single out the upper quantiles (0.9, 0.95, 0.99) as most important for decision making (arXiv:2502.19086). That is the right frame for epidemic forecasting, where the decision-relevant question is usually the tail: how bad could the next four weeks be?

The pre-citation audit: four things to demand before a dashboard number

Combining the paper’s transparency affordances with the scoring critique and the LLM failure modes documented in adjacent work, I would hold any LLM-guided epidemic forecast out of situational reports until it clears four checks. This is my inference from the cited evidence, not a finding of the paper:

The season-length requirement in row two deserves emphasis. FluSight’s operating model already lets many teams submit weekly forecasts for several targets in real time against weighted influenza-like illness data (arXiv:1910.07084), so the infrastructure to score a new entrant alongside incumbents exists. A single season is a modest ask compared to the cost of citing a forecast that turns out to be miscalibrated during the exact weeks it mattered.

LLM-specific failure modes to test first

Some of what can go wrong is generic to automated modeling, and some is specific to having a language model in the loop. The most useful checklist comes, candidly, from the limitation sections of other LLM forecasting papers. The authors of LLM-Mixer, a time-series system, list six limitations, four of which read:

“Prompt Engineering: Model performance depends on prompt quality and domain expertise for optimal prompt design, which may limit accessibility for non-experts. Out-of-Distribution Robustness: When training and test data distributions differ significantly, the fixed prompt approach may not adapt effectively to distributional shifts. Limited Classical Baseline Analysis: Our evaluation focuses primarily on deep learning methods and would benefit from comprehensive comparison with statistical approaches like ARIMA and exponential smoothing. Data Leakage Potential: While we implement protocols to prevent information leakage, the prompt-based approach requires careful validation to ensure fair comparison” (arXiv:2410.11674).

Each maps onto a question for ERA. Prompt sensitivity becomes judge sensitivity: the paper tested one in-loop judge design, so the search’s steering signal is a single point in a design space. Distribution shift is endemic to surveillance, where reporting pipelines, testing behavior, and case definitions change mid-season. Missing classical baselines is a question worth asking before citing anything from the generated pool: which simple statistical models (seasonal baselines, ARIMA-style smoothers) did the pool beat, and by how much? Leakage is the reason the “leak-proof” claim needs the timestamped submission book rather than the authors’ assurance.

A second adjacent paper shows how the field is responding to LLM unreliability in forecasting. ScenarioDiff’s authors deliberately keep the LLM away from the numbers:

“In contrast, ScenarioDiff uses LLMs for producing contextual information rather than direct numerical generation: the agents summarize historical evidence, produce scenario descriptions, and extract anchor points, while a diffusion forecaster handles probabilistic generation” (arXiv:2608.17164).

Critically, they validate the LLM-generated context against misalignment controls, shuffled or random text, before treating it as signal. That control is the right test for a subtler ERA risk: if the LLM writes model code that encodes plausible-sounding epidemiological reasoning (an invented covariate, a spurious seasonal structure), the model can score well on a validation window for the wrong reasons. A shuffle-style check, asking whether the model still performs when its narrative rationale is scrambled or removed, is cheap insurance against confident nonsense. These failure modes come from general time-series settings, not from epidemic forecasting specifically, so treat them as plausible risks to probe, not measured defects of ERA.

What a health department must stand up

Even a validated forecast lands in an organization, and the organizational evidence is cautionary. Public-health decisions are made under distribution shift with batched, simultaneous resource allocations, and in one bandit-algorithm simulation study, “greedy resource allocation strategies… often perform worse than random allocation as they fail to notice outbreaks in new areas” (arXiv:2110.13306, abstract-level evidence). A forecast that is accurate on average but misses a newly emerging location feeds exactly the greedy allocation pattern that study warns against, which is another reason the upper quantiles and turning-point behavior matter more than average leaderboard rank.

Coordination is the second operational risk. New Zealand’s COVID-19 experience with digital tools showed that “when efforts are not well co-ordinated, fragmentation in system design and user experience can negatively impact the public health response” (arXiv:2009.06279, abstract-level). An LLM forecaster bolted onto an existing surveillance workflow without a named owner for scoring, auditing, and decommissioning is fragmentation waiting to happen.

The monitoring plumbing is not trivial either. A case study from the PETs for Public Health Challenge combined differentially private synthetic data with public-health and mobility datasets to produce “six reusable tools and frameworks supporting diagnostic nowcasting (e.g., Hotspot Detection, Pandemic Adherence Monitoring) and predictive forecasting” (arXiv:2603.13444, abstract-level). The lesson is that useful forecasting infrastructure takes deliberate engineering investment; it does not fall out of a modeling breakthrough. None of this is a reason to refuse LLM-generated forecasts. It is a reason the validation question is an organizational one, not just a statistical one.

When an LLM-generated forecast belongs in a situational report

The honest reading of arXiv:2605.16238 is that it is a claim to score, not evidence to cite. The authors built a system whose outputs are unusually auditable, ran what they describe as a prospective leak-proof evaluation, published code and submissions under Apache 2.0, and disclosed their system’s known biases in their own discussion. That is more transparency than most forecasting claims arrive with. It is still a single team’s unreplicated account of its own system, scored under a Weighted Interval Score variant the paper itself describes as proper, and the parity claim remains the authors’ self-reported summary.

So the decision rule I would apply: a generated forecast earns a place in a situational report only after your department, or a body you trust, has run the four-part audit above across at least a full season, under a proper scoring rule, with the executable code retained and the turning-point behavior measured. Before that, ERA’s weekly submissions belong in a watched evaluation folder alongside your own ensemble, not in the numbers you brief on. The cost of that patience is one respiratory season of parallel running. The cost of skipping it is citing a number that cannot be scored until after the decisions it shaped are already made, which is the one outcome a forecast exists to prevent.

Frequently Asked Questions

Where can I find the ERA model code and forecast submissions?

Model source code and weekly forecast submissions are public under Apache 2.0 at the project’s GitHub repository, a contrast the authors draw explicitly with black-box AutoML.

What specific failure mode do the ERA authors concede regarding epidemic trends?

“models tend to underestimate the rate of increase as the epidemic is rising and the rate of decrease as it falls… Forecasts generated by ERA are no exception” (arXiv:2605.16238).

Why is the multibin log score used in past CDC competitions problematic?

“We showed that the multibin log score used in the CDC FluSight competitions incentivizes hedging, i.e. tuning forecast distributions in a specific way before submission. While we strongly doubt that participants have consciously tried to game the score, it is possible that this happens unintentionally” (arXiv:1910.07084).