On July 6, 2026, arXiv:2607.04919 posted the first systematic break-even analysis of pretrained time series foundation models against the classical baselines practitioners reach for by default. Across 30 benchmark datasets, Chronos, Moirai, and Lag-Llama beat Naive, ETS, ARIMA, and XGBoost at every training fraction on exactly half the data. On the other half, the answer depends on how much history you have, how strong your seasonality is, and whether you fine-tune.
When Does GPU Infrastructure Pay Off?
The question is not whether foundation models beat classical forecasters, but at what data volume the GPU cost of running them buys accuracy that ARIMA or XGBoost cannot match. Time series foundation models such as Chronos (Amazon), Moirai (Salesforce), and Lag-Llama are pretrained on large heterogeneous corpora and sold as zero-shot drop-ins: skip the per-series modeling, skip the feature engineering, point the model at your series and forecast. Leaderboards, where these models post strong zero-shot numbers, do not answer when that upgrade stops paying.
The break-even analysis reframes model selection as a regime question. Rather than asking which model tops a benchmark, it asks, for each dataset, how much history a foundation model needs before it overtakes the classical baseline you would have reached for anyway. That shifts the decision onto properties of your data rather than the rank of the model. The deliverable is a set of curves and thresholds, and the threshold matters more than the headline win rate. Foundation models are pitched as a drop-in upgrade, but the break-even math exposes a boundary: below a certain history length, or for certain series structures, pretraining stops paying and you spend GPU budget for worse-than-baseline accuracy.
How Was the Benchmark Designed?
The study ran three foundation models in both zero-shot and LoRA fine-tuned configurations against four classical baselines (Naive, ETS, ARIMA, XGBoost) across 30 benchmark datasets, each tested at six training fractions spanning 2% to 100% of available data.
The six-fraction sweep is the part most foundation model papers omit. They typically report at a single fraction, usually full data, or zero-shot only. Sweeping from 2% to 100% traces a break-even curve for each dataset, which is the shape a practitioner needs: the point at which the pretraining advantage outweighs what a classical model can extract from raw local history. Classical methods cannot import learned patterns from other series, so their accuracy depends entirely on how much of the target series they can see. Foundation models can import patterns, which is why their advantage concentrates in the low-data regime and erodes as history grows. The baseline choices matter too: Naive is the floor any model must beat to justify existing, ETS and ARIMA are the econometric workhorses, and XGBoost with lagged features is the strong tabular baseline most production teams actually deploy.
Which Datasets Favor Foundation Models, and Which Favor Classical Methods?
The 30 datasets split into three regimes. Foundation models win at every training fraction on 15 of them, meaning GPU deployment is unconditionally justified regardless of how much history you have. The remaining 15 divide into two groups where the answer is conditional.
On 6 datasets, classical methods surpass zero-shot foundation models with as little as 2% of training data, between 21 and 2,768 samples. On these, pretraining never recovers: the foundation model is worse than a classical baseline at any data volume the study tested. On the remaining 9 datasets, the foundation model eventually wins, but only after a break-even threshold that ranges from 24 to 8,361 samples. Below that threshold you spend GPU budget for worse-than-baseline accuracy.
| Regime | Datasets | Behavior |
|---|---|---|
| Unconditional foundation-model win | 15 of 30 | Beats classical at every training fraction |
| Classical dominance | 6 of 30 | Classical beats zero-shot with as little as 2% of data |
| Break-even threshold | 9 of 30 | Foundation model wins only after 24–8,361 samples |
The asymmetry is the headline. Half the datasets make foundation models a free upgrade. A fifth make them a strict loss. The middle third make them a bet that only pays once you clear a data-volume bar you have to measure first.
When Is Zero-Shot a No-Brainer?
One rule needs no model training: if your training set has fewer than 700 samples and seasonality is non-negligible, use a foundation model zero-shot and skip fine-tuning. According to the paper, this single rule resolves 10 of the 30 deployment decisions immediately.
The mechanism is straightforward. Foundation models carry learned seasonal patterns from pretraining. A classical method such as ARIMA or ETS must estimate seasonality from whatever short history you hand it, and with fewer than 700 points that estimate is noisy. The pretrained model imports a prior that the classical model has to rebuild from scratch. When seasonality is present and history is short, the imported prior wins. When history is long enough, the classical model’s locally fit estimate catches up and then surpasses the imported one, which is the inflection the break-even curves capture.
When Does Fine-Tuning Make Things Worse?
LoRA fine-tuning, the standard response to a zero-shot model that underperforms, can actively degrade accuracy on short series. The common practice assumes fine-tuning always improves on zero-shot. The break-even data contradicts that assumption at the low-data end.
The intuition is that with little target data, fine-tuning overfits to noise faster than it transfers useful structure. The pretrained weights encode a useful prior; a few hundred noisy samples can overwrite that prior with local artifacts. The recommendation is concrete: in the regime where zero-shot wins, skip fine-tuning entirely. The instinct to fine-tune “to be safe” is the wrong instinct in exactly the regime where people are most likely to reach for it, because short series are where zero-shot looks most attractive and fine-tuning looks least risky.
How Do You Decide Without Running the Full Comparison?
The paper proposes a two-step decision framework: first compute dataset length and seasonality strength, then run a brief 5 to 10% pilot only if the simple rule does not already resolve the case. The point is to decide before committing to full infrastructure.
For the cases the n_train < 700 rule does not cover, four dataset features motivate mechanistic hypotheses about which regime a given series will fall into. The authors are explicit that reliable automated prediction across the full benchmark set remains an open problem: you can narrow the decision, but you cannot yet fully automate it from dataset features alone. The pilot step exists because the remaining uncertainty has to be resolved empirically, and a 5 to 10% pilot is cheaper than a full deployment that turns out to be a loss.
What Does This Mean for Production Forecasting Teams?
Foundation models are a regime-dependent bet, and the break-even curves are how you tell which regime you are in. The practical shift is that model selection moves upstream into data characterization, before any model runs. Teams that deploy foundation models as blanket replacements for their classical forecasters will overspend on the 20% of datasets where classical methods win at any data volume, and they will overspend again on the datasets where fine-tuning erodes accuracy rather than improving it.
The macroeconomic case study makes the regime dependence visceral. Across India’s exports and imports over a 26-month volatile test window, Chronos zero-shot edged out ARIMA on exports (RMSE 2788.72 versus 2807.77) but lost badly on imports (RMSE 4442.13 for ARIMA versus 6005.02 for Chronos). Same country, same forecaster, same window, opposite verdicts. The export win is marginal, about 19 RMSE; the import loss is roughly 1,563 RMSE, two orders of magnitude larger. The export series apparently fell in a regime where the pretrained prior helped; the import series did not. That is the whole argument compressed into one country’s trade data.
Independent benchmark harnesses are beginning to standardize the comparison. One community harness evaluates ARIMA, Prophet, LSTM, and XGBoost against Chronos across point accuracy, probabilistic calibration, computational cost, and few-shot learning curves. These repos do not yet offer the regime-based decision framework, but they make the raw comparison reproducible for teams that want to fit their own break-even curves.
The honest summary for a production team: foundation models are not a replacement for classical forecasting, they are an addition to the toolkit that pays off in a measurable subset of cases. The subset is defined by your data, not by the leaderboard. The n_train < 700 rule resolves the easy cases for free; the rest require a short pilot and the willingness to admit that, on a fifth of datasets, the GPU does not help.
Frequently Asked Questions
Does the n_train < 700 rule work for every time series?
No. It applies only when seasonality is non-negligible. If your series is trend-only or has weak seasonal structure, the rule does not resolve the decision and you should move to the 5-10% pilot step. The authors identify four dataset features that motivate hypotheses for these remaining cases, but they stress that fully automated prediction across all 30 datasets is still an open problem.
What if I cannot compute seasonality strength internally?
Use a standard spectral density estimate or check autocorrelation at the dominant lag. If you lack that tooling, treat the absence of a clear seasonal peak as negligible seasonality and run the pilot anyway. The framework is meant to let the simple rule handle the easy cases while the pilot resolves the rest, not to force every team to build a custom feature extractor before they can decide.
How is this break-even study different from vendor leaderboards?
Vendor benchmarks usually report a single training fraction, often full data or zero-shot, and emphasize aggregate rank. The break-even study instead sweeps six fractions from 2% to 100% to produce a per-dataset curve. Community benchmark suites such as mayank02raj/TS-Foundation-Benchmark add point accuracy, calibration, and cost metrics, but they still do not replace the pilot because the aggregate win rate can hide whether your series sits in the classical-dominance regime.
Can fine-tuning ever be worse than doing nothing at all?
Yes. On short series, LoRA fine-tuning can overwrite the pretrained prior with noise from a few hundred samples. The hidden cost is that you burn GPU time on fine-tuning and still end up below the zero-shot foundation model, and sometimes below ARIMA or ETS. Treat fine-tuning as a measured experiment, not a safe default.
Will this framework still matter as newer foundation models ship?
New models may shift the thresholds or expand the unconditional-win group, but the 20% classical-dominance regime is likely to persist because it reflects series structures where pretraining imports priors that do not match local dynamics. Chronos, Moirai, and Lag-Llama will be superseded, yet the workflow, characterize data, pilot, then provision, stays intact. Track updated benchmarks, but fit your own curve rather than relying on headline rank.
