Time-Series Foundation Models vs Classical ML for Building Energy Forecasts

If you run building energy forecasts and you have a few years of smart-meter history, the cheapest correct answer is still a tuned gradient-boosting model on your own meter data. That is the clearest practical finding from a head-to-head arXiv study datelined 2026-10-08, which put Google’s TimesFM 3.0, run zero-shot with covariates, against XGBoost for day-ahead electricity forecasting in a university building. The foundation model won on aggregate error, cutting mean absolute error roughly 43% versus a weekly seasonal-naive baseline, the study reports, but paired day-by-day comparisons did not establish a clear MAE advantage over XGBoost. Zero-shot foundation forecasts are now good enough to be worth testing on your meters; they are not yet good enough to replace a tuned local model by default.

That distinction matters more than the headline number, and the rest of this article is about turning it into a decision you can run yourself: a backtest protocol, the real plumbing cost of “zero-shot,” the documented failure mode that should keep you skeptical, and a routing rule for when each option earns its place.

The Delft study, number by number

The study used hourly whole-building electricity consumption from a modern, well-insulated teaching building on the Delft campus of The Hague University of Applied Sciences (THUAS), evaluated over 57 days including closures, according to the paper’s HTML version. Eleven configurations were benchmarked (Ridge, XGBoost, two Prophet variants, two LSTM configurations, three TimesFM configurations, and two seasonal-naive baselines); three decide this question: TimesFM 3.0 with covariates, XGBoost, and the weekly seasonal-naive forecast (this hour last week).

The headline result, study-reported: “TimesFM 3.0 with covariates achieves the lowest overall errors, reducing mean absolute error by approximately 43% relative to weekly seasonal-naive forecasting.” (abstract) That reduction is measured against seasonal-naive, not against a tuned model. Against XGBoost, the paper is more careful: “paired daily comparisons do not establish a clear mean-absolute-error advantage for TimesFM.”

The authors also surface a tradeoff that aggregate MAE hides. “XGBoost retains an advantage when larger non-workday errors are the priority: its non-workday RMSE is 6.13 kWh lower than that of TimesFM 3.0 with covariates, despite their MAEs differing by only 0.18 kWh.” In other words, the two models produce nearly identical average misses on non-workdays, but TimesFM’s misses include larger outliers. The paper’s own conclusion: “The preferred error profile therefore depends on the operating period and the consequences of larger misses.” If your building’s weekend HVAC setbacks, demand charges, or battery dispatch decisions are sensitive to occasional large errors, that RMSE gap is the number to weigh, not the aggregate MAE.

One more detail worth keeping: this is not the first model comparison on this building. The paper notes a prior THUAS evaluation “found the lowest workday MAE with Prophet and the lowest MAPE and mean squared error with XGBoost.” The ranking of classical methods on the same meter has already shifted once depending on the metric. Expect it to shift again.

A backtest protocol you can run on your own meters

The study’s most reusable output is not its ranking but its shape. You can rerun the same contest on any meter with a few years of hourly history, and the protocol has five parts that each change the answer if you skip them.

1. Horizon. Day-ahead, hourly: predict all 24 hours of tomorrow. This is the horizon building operators and small utilities actually schedule against, and it is what the Delft study evaluated.
2. Baselines first. Weekly seasonal-naive (same hour, previous week) and a flat naive forecast. Any model that cannot beat seasonal-naive on a building meter is not a model, it is overhead. The headline reduction is a statement about clearing this bar, and clearing it is the entry fee, not the prize.
3. A tuned local model. Gradient boosting with calendar features (hour, day-of-week, holiday flags) and weather covariates. Tuning does not have to be elaborate: a community LightGBM-versus-Moment energy forecasting project found its final parameters with a custom grid search over roughly 50 combinations, landing on learning_rate 0.05, num_leaves 15, max_depth 5, lambda_l1 1.0, and 1,000 estimators (grid-search notes). That sets the bar a foundation model must clear.
4. Seasonal splits. Evaluate across seasons and, critically, across operating regimes: workdays, non-workdays, closures. The Delft study’s 57-day window included 12 weekend days and 11 holidays, and it was across those non-workdays as a group, not closures alone, that the XGBoost RMSE advantage appeared. A split that averages regimes together will hide exactly the errors that cost money.
5. Paired daily scoring. Score each day separately and compare models on the same days, rather than comparing aggregate error across the whole window. This is the step that turned “TimesFM wins” into “TimesFM does not clearly beat XGBoost” in the study. Aggregate metrics can be carried by a stretch of easy days; paired comparisons tell you which model you would rather have on any given Tuesday.

The acceptance rule I would use, and this is inference from the evidence rather than a measured finding: switch a meter to zero-shot foundation forecasts only when paired daily backtests on that meter beat both weekly seasonal-naive and your tuned local model on the error metric that matches your operating cost (MAE for average miss size, RMSE for exposure to big misses).

The hidden cost of “zero-shot”

Zero-shot describes the training you skip, not the data engineering you skip. In the Delft study, TimesFM 3.0 consumed roughly 30 covariates, seven of them weather variables, alongside the electricity history, with an 8,760-hour context window (a full year of hourly data) feeding the model, per the paper’s description of its weather-source ablation. That ablation held the consumption history, the other 23 covariates, the forecast windows, and all TimesFM settings fixed, replacing only the seven weather covariates in both the context and the target hours.

Two consequences follow. First, covariate plumbing does not disappear when you stop training. You still need a weather source you trust and aligned timestamps. The selected TimesFM configuration consumed an 8,760-hour context, but the paper’s extended evaluation warmed up from 168 hours (study) and grew from there, so a year of history is what the tuned setting consumes, not a gate in front of the first forecast. Second, the study did treat the weather source as a tunable, and it measured the stakes: substituting ERA5 reanalysis or archived ECMWF IFS forecasts for KNMI station observations changed the selected configuration’s MAE by at most 0.17 kWh (paper). But that experiment substituted one complete weather source for another; it never removed weather, so it cannot rank choice of product against presence of weather. On the pooled numbers, the full covariate set barely moved TimesFM: 20.01 kWh MAE target-only versus 19.97 kWh with covariates (Table 6). What the covariates did buy was holiday accuracy, with MAE falling from 34.40 to 24.76 kWh (paper). If you build the covariate pipeline for a building like this, the paper’s own numbers say you are building it for the closures, not for the average day.

What about runtime? The paper measured it. TimesFM 3.0 with covariates took a median 0.92 seconds to forecast 24 hours once loaded, with 4,402.73 MiB of peak process memory and 2,470.39 MiB of peak GPU memory, plus roughly two seconds to load (Table 11); XGBoost refit in a median 0.84 seconds per day, inferred in about 0.0012 seconds, and peaked at 378 MiB (same table). Neither time is a burden on a daily cycle, but the memory difference is what you feel if one box serves many meters. And the paper’s sharpest deployment caveat is not compute at all: the evaluated TimesFM 3.0 weights are released for non-commercial, non-production research use, and the authors state their results “do not establish permission to deploy this checkpoint in production.”

When zero-shot fails: the alignment problem

The strongest reason to keep the paired-backtest gate is independent evidence that zero-shot accuracy is fragile in a specific, predictable way. A separate arXiv study examining how “foundational” time-series foundation models really are reports that SAMFormer, a model trained entirely from scratch with fewer than 50,000 parameters, outperformed a fine-tuned TimesFM (over 500 million parameters) on a two-year daily electricity forecasting task. Those experiments ran on a Tesla V100 GPU; the authors characterize SAMFormer as lightweight and efficient to train on consumer-grade GPUs (same paper).

The mechanism the authors identify is the one to remember: “TSFMs’ zero-shot performance is highly sensitive to the alignment between the statistical properties of the pretraining and target domains. When this alignment is weak, we observe substantial drops in generalization, even across domains that might appear related.” Electricity data to electricity data is not automatically good alignment; the statistical texture (daily versus hourly resolution, amplitude distribution, regime structure) is what matters.

This is not an argument that foundation models are overhyped. It is an argument that a model’s pretraining corpus is a bet about your data, and sometimes the bet loses to a small model trained on the actual thing. The FactoryNet industrial pretraining corpus illustrates the corpus side of the bet: 113 million datapoints across 57,000 task executions with 27 annotated anomaly types is a serious industrial time-series corpus, and it looks nothing like a building meter. Which pretraining distributions actually contain something like your meter’s regime is an empirical question your backtest answers.

Routing: which option for which meter

Putting the Delft study, the SAMFormer result, and the protocol together, the decision splits on two axes: how much history the meter has, and how much tuning capacity you have per site. The table reflects evidence-supported routing; the Moment figures come from a community project measured on city-wide Chicago data, not a building meter. One constraint applies to every row that names TimesFM 3.0: the evaluated checkpoint is licensed for non-commercial, non-production research use (paper), so treat the zero-shot route as a backtest candidate, not a deployable default for facility operations.

One honest gap in that table: “many sites” is where zero-shot should shine, because training cost scales with sites and inference does not, but no study cited here measures a multi-site deployment. The Delft study is one building. The routing recommendation for fleets is an extrapolation from the mechanism (no per-site fitting) plus the alignment caveat, not a measured result.

Adaptation research worth watching

Two October 2026 papers sketch the middle ground between zero-shot and full retraining, and both are worth tracking if you manage a meter fleet, though neither is a shipping tool you can adopt today.

ScaleIn is a scale-invariant pre-training loss that, applied during TSFM pre-training, improved MASE and WQL on average by 18.8% on GIFT-Eval and 21.9% on M-competitions (authors’ reported figures). The practical implication is not the numbers; it is that foundation-model accuracy is still moving at the pre-training-loss level. The TimesFM you evaluate this quarter is not the model you will be offered next year, so version-pin your backtests and expect to rerun them.

AdaCast takes the other end: keep the backbone frozen, inject instance-specific low-rank updates generated per series. The authors report it “consistently outperforms static adaptation baseline in in-domain forecasting and improves zero-shot generalization to held-out datasets across domains,” with the lowest average MSE (0.2105) and MAE (0.2576) across six datasets. If that holds up, it describes exactly what a campus analyst with one to two years of data per building wants: foundation-model priors, adjusted cheaply to each meter. Today it is a research result with no verified shipping implementation, so it belongs in the watch pile, not the routing table.

Where this will not transfer

Every building-specific number in this article comes from one modern, well-insulated teaching building in Delft, evaluated over 57 days, with no independent replication. An older building with erratic occupancy, a different climate, sub-metered circuits instead of whole-building feeds, or a meter with gaps and estimation flags could flip every ranking here. The documented sensitivity of zero-shot performance to pretraining/target alignment is precisely the mechanism by which that flip would happen.

So the consequence is real but bounded. If zero-shot forecasts hold up on your meters, the economics of building energy forecasting shift from per-site model training to covariate and context selection: your effort moves into weather-data plumbing, calendar hygiene, and keeping a year of clean history per meter. The way to find out is not to trust the Delft ranking or this article’s routing table. It is to run the five-step backtest on your own data, score it day by day, and let your meters vote. Until they do, the tuned local model stays the default, not because foundation models lost, but because “no clear paired-daily advantage” is not a reason to replumb a working pipeline.

Frequently Asked Questions

What is the licensing restriction for the TimesFM 3.0 checkpoint evaluated in the Delft study?

the evaluated TimesFM 3.0 weights are released for non-commercial, non-production research use, and the authors state their results “do not establish permission to deploy this checkpoint in production.”

How much memory does TimesFM 3.0 use compared to XGBoost for forecasting?

TimesFM 3.0 with covariates took a median 0.92 seconds to forecast 24 hours once loaded, with 4,402.73 MiB of peak process memory and 2,470.39 MiB of peak GPU memory, plus roughly two seconds to load (Table 11); XGBoost refit in a median 0.84 seconds per day, inferred in about 0.0012 seconds, and peaked at 378 MiB (same table).

What specific error metric advantage does XGBoost have over TimesFM 3.0 on non-workdays?

XGBoost retains an advantage when larger non-workday errors are the priority: its non-workday RMSE is 6.13 kWh lower than that of TimesFM 3.0 with covariates, despite their MAEs differing by only 0.18 kWh.