Do Tabular Foundation Models Beat XGBoost in Finance? A Routing Guide

Tabular foundation models now beat tuned XGBoost on at least one public credit-default dataset and, according to a new accounting research note, on a financial forecasting task, but the honest answer to “do they beat XGBoost in finance” is: sometimes, on small-to-midsize tables, with caveats that decide the routing. The evidence as of November 2026 supports a champion, challenger pattern, not a migration. Keep the tuned gradient-boosted model as champion. Pilot a tabular foundation model (TFM) where labeled rows are scarce, retraining and tuning cycles dominate your pipeline cost, and calibrated probabilities matter more than per-row latency. Keep the GBDT where latency or throughput binds, where fairness-constrained training or tree interpretability is required for a regulated decision, or where customer data cannot leave your environment.

That routing rests on a specific, measurable pattern: in an independent credit-risk benchmark, TabPFN’s lead over XGBoost was +0.056 AUC at 100 training rows but only +0.023 at roughly 26,000. The advantage is real and shrinking with data volume. Everything below is about where your workload sits on that curve, and on the other axes the headline number hides.

What the new finance paper actually claims

The news hook is arXiv:2611.13550, “Tabular Foundation Models in Financial Predictions”, submitted 10 November 2026 by Allen Huang. The paper introduces TFMs to accounting research through one benchmark task: predicting firms’ next-year earnings changes. Its headline, per the abstract:

“We compare Google Research’s TabFM with a fine-tuned open-weight large language model developed in recent accounting research and with several tree-based models. TabFM is more accurate than all comparison models. It is also faster and less computationally intensive than the large language model. Several tree-based specifications produce predictions more quickly than TabFM, so choosing between them involves a trade-off between accuracy and speed.”

Two details matter for routing. First, the accuracy, speed trade-off is in the paper itself: trees won on prediction speed even as TabFM won on accuracy. Second, the authors scope their own result:

“We conclude that TFMs have promise as a new tool for forecasting in accounting and finance. However, our evaluation is based on a single benchmark task, and no class of models is likely to dominate in all settings.”

A caution before anyone cites this paper in a model committee memo: the public abstract reports no numeric results, so there are no scores to quote. The paper also notes its TFMs are “pretrained on synthetic tabular data, so they avoid look-ahead bias from memorizing firms’ outcomes,” a design point that matters in finance, where a model pretrained on public web text may have seen the outcomes it is asked to predict. That concern is corroborated by the tooling ecosystem: Prior Labs’ current default, TabPFN-3.5 is likewise “pretrained purely on synthetic data.”

Treat the paper as a domain-credibility signal: a finance-specific task where a TFM beat tree baselines, with a speed caveat attached. Do not treat it as a measured result you can put in a validation document.

The independent cross-check: TabPFN vs tuned XGBoost on credit default

The quantitative evidence comes from a community benchmark on a public Kaggle credit dataset (32,581 raw rows, 11 features, 21.9% default rate; 25,927 training rows after cleaning, with a fixed 20% stratified test split). On that dataset:

• TabPFN, accessed via API with no task-specific training: ROC-AUC 0.972, Brier score 0.040.
• XGBoost, tuned with a 50-candidate randomized search and 5-fold cross-validation: AUC 0.953, Brier 0.066.
• Balanced random forest: AUC 0.936. Balanced logistic regression: AUC 0.871.

The sample-size scaling runs are the useful part for routing. Per the benchmark, “TabPFN had the best ROC-AUC at every training size, in all 12 runs. The lead is largest with very little data (+0.056 AUC at 100 rows). Tuning XGBoost barely helps (0.950 → 0.953 AUC); TabPFN still leads by 0.020.” At 1,000 rows the lead was +0.023; at 10,000 rows +0.025; at the full 25,927 rows +0.023.

Read that two ways. The TFM lead is robust across sizes on this dataset, which pushes back on the assumption that TFMs only win on tiny tables. But the largest margin by far is in the small-data regime, and no source here measures behavior beyond roughly 26,000 training rows on a financial task.

The benchmark’s authors are explicit about its limits, and so am I: one public dataset, one test split, only XGBoost tuned (no CatBoost or LightGBM), and a modest tuning budget. Their own note: “Results may not transfer to other credit portfolios.” This is a real measurement, not a universal law.

Axis 1 and 2: How many labeled rows do you have, and does the table fit?

Labeled-row count is the first routing question because the evidence cuts differently at each end. Below roughly a thousand labeled rows, the credit benchmark’s +0.056 AUC margin at 100 rows is the strongest single number in this brief for piloting a TFM. Scarce-label workloads are common in finance: a new fraud typology, a niche lending product, an early-stage collections segment. If you are tuning a GBDT on 300 labeled defaults, the evidence favors a TFM challenger.

Table size is a hard constraint in the other direction. Per the TabProbe paper citing TabICL results, “The current family of TFMs is limited by context size, however, still reaching up to 500K rows (Qu et al., 2025) on affordable resources. This limit, despite being significantly higher than the earlier TFMs (e.g., TabPFN v1), can prevent learning rules in very large tables.” A 500K-row ceiling covers many credit and marketing scorecards, but not high-volume transaction fraud tables or large deposit-behavior datasets, at least not without subsampling, which changes what the model sees and how you validate it.

Axis 3: Latency and throughput

The finance paper’s own caveat applies broadly: “Several tree-based specifications produce predictions more quickly than TabFM.” Mechanistically, TFM inference cost scales with the size of the context you supply; the TabSwift paper measures TFM runtime as wall-clock time per query given the support set, growing with dataset complexity (rows times features), while noting “tree-based ensembles such as gradient-boosted decision trees (GBDTs) remain strong and widely used baselines for tabular prediction.”

If your scorecard prices thousands of applications per hour against a strict latency budget, a GBDT’s cheap per-row inference is a structural advantage. If you score a batch of a few hundred cases nightly, the accuracy, speed trade-off the finance paper describes may be worth paying. This is a workload question, not a model-quality question.

Axis 4: Calibration, thresholds, and money attached to probabilities

For credit pricing and provisioning, AUC is not the metric that moves money; calibrated probabilities are. Here the credit benchmark found a genuine difference:

“TabPFN’s probabilities are well calibrated out of the box. The class-weighted models overpredict default (e.g. XGBoost predicts ~45% where ~18% default) and would need recalibration before their probabilities are used for pricing or provisioning. Their higher Brier scores partly reflect that weighting, not only weaker models.”

Two implications. First, out-of-the-box calibration is a real operational advantage when probabilities feed expected-loss calculations directly. Second, the comparison is partly a pipeline artifact: class weighting is a common GBDT practice for imbalanced default data, and it distorts probabilities until you recalibrate. The same benchmark found threshold tuning helped XGBoost more (F1 0.810 to 0.832) than TabPFN (0.874 to 0.877), meaning the tree pipeline had more headroom from post-processing. A well-run GBDT pipeline with recalibration closes some of the gap the raw numbers suggest. Price that engineering work into the comparison rather than treating the raw Brier scores as final.

Axis 5: What “no retraining” actually changes

The economic case for TFMs is not the accuracy headline; it is the removal of the train-tune-redeploy cycle. Per the credit benchmark, a 50-candidate XGBoost tuning search moved AUC only from 0.950 to 0.953. If your team runs those cycles quarterly across a dozen scorecards, a model that predicts in-context with no task-specific training changes where the money goes.

But it changes the cost location, not the total. Three mechanisms from the evidence:

Context construction replaces feature engineering. TFMs appear to behave like retrieval systems over the rows you supply. The paper on TFM generalization “argued for a retrieval-focused interpretation of TFMs rather than one based on standard transfer learning or Bayesian inference.” Which historical rows you put in context, and how they are sampled, becomes the lever that matters. That is a new skill and a new failure surface: a badly constructed context is the TFM equivalent of a badly engineered feature set.

Drift handling changes shape. A frozen-weight TFM adapts to evolving data only by updating its retained context. The Cure stream-learning paper found “a TFM with bounded context management can be a strong stream learner compared with online tree and ensemble methods,” while noting “Cure adapts to evolving streams only by updating the retained context.” Your drift review shifts from “when do we retrain” to “what does the retained context currently represent, and when did it last turn over.” That is a different monitoring discipline, and your model-risk team will want it documented as such.

“No validation set needed” is an input advantage, not a validation substitute. The RDB foundation model paper frames the goal as retaining “applicability across unseen RDBs, ideally with no retraining required for each new predictive task,” and notes ICL models can repurpose held-out data as context. Using validation rows as extra context can improve predictions; it does not remove the need to evaluate on data the model has not consumed as input. Any vendor framing that blurs this distinction should worry your validators.

Axis 6: Model risk, where the GBDT still owns the argument

Three constraints keep regulated workloads on trees today.

Interpretability. The credit benchmark’s own caveat: TabPFN “is also harder to interpret than tree models, which matters for regulated credit decisions.” Mature tooling exists for explaining tree ensembles to reviewers and customers; the TFM equivalents are younger.

Fairness-constrained training. This is the sharpest gap. Per the FairTFM paper: “While fairness-aware training has been extensively studied in classical machine learning, these approaches do not readily extend to the ICL paradigm, where predictions must be produced in a single forward pass without task-specific optimization.” Techniques like Exponentiated Gradient applied to logistic regression, random forest, or XGBoost have no direct analogue for a model you cannot train on your task. The same paper notes TFMs “can exhibit biased outcomes similar to traditional models,” and its own remedy, incorporating fairness into pretraining across 132 tasks, is a research direction, not a control you can apply on demand. If fair-lending constraints are part of your approval path, that is a routing decision made for you, for now.

Data residency. TabPFN in the benchmark “runs through the Prior Labs cloud API. Data leaves your machine, which matters for real customer data.” Local serving options exist in the ecosystem, but verify the deployment path for your chosen checkpoint before a pilot touches production data.

A cautionary result for the fraud workflow around the model: in a layered PaySim fraud pipeline, an LLM investigation agent with access to explanations, graph context, and reference cases “underperforms direct thresholding of the classifier it relies on, reaching 65.0% accuracy against 71.7% on a balanced 60-case sample.” A coherent rationale is not evidence of a better decision. If you bolt agentic review onto any classifier, TFM or GBDT, measure it against the trivial baseline.

The counter-evidence is not a footnote

Tree ensembles have not surrendered. The Robust TFM paper cites survey work finding “a significant gap in performance remained” between boosted trees and deep learning on structured data (Grinsztajn et al. 2022), and builds its own training signal on an “optimality gap” measured against XGBoost, CatBoost, and Random Forest, a construct that exists precisely because TFMs still trail those baselines on some datasets. RTFM reports improving TabPFN V2 by up to 6% mean normalized AUC, which is another way of saying the unmodified TFM left that much on the table.

On finance tasks specifically, FinBench (default, fraud, churn) found fully fine-tuned foundation models beat tree-based methods on average F1, but prompted LLMs without task-specific tuning “are not good classifiers compared with other baselines.” Fine-tuning is a different regime from in-context prediction, with different costs; do not let a fine-tuning result launder a zero-shot claim.

And the pre-foundation-model baseline condition still stands: the Revisiting Deep Learning Models for Tabular Data benchmark concluded “there is still no universally superior solution among GBDT and deep models.” Every result since, in both directions, is consistent with that.

The routing table

What would change this recommendation

The finance-specific evidence is thin, and the routing guide should be held loosely where it rests on thin ground. The public finance note abstract reports no numeric results; when its full text lands with protocol, baselines, and numbers, the earnings-change claim may strengthen or shrink. The quantitative credit result is one public Kaggle dataset with one test split and XGBoost-only tuning; a CatBoost or LightGBM baseline with a larger search budget could close the gap, and neither source touches proprietary bank portfolios, marketing response models, or zero-shot TFMs on fraud. No source here documents regulator acceptance of a no-retraining model.

I would run this as a six-to-eight-week champion, challenger on one scorecard where labels are scarce and the table is small, with drift review and validation documentation written for the context-update mechanism rather than borrowed from the retraining playbook. If the challenger wins on calibrated metrics and survives your model-risk review, expand. If a lost fairness control or a data-residency blocker applies to the workload, the decision is already made, and the GBDT keeps the chair.

Frequently Asked Questions

How much better is TabPFN than XGBoost on small credit datasets?

TabPFN’s lead over XGBoost was +0.056 AUC at 100 training rows but only +0.023 at roughly 26,000. The advantage is real and shrinking with data volume.

Can tabular foundation models handle very large tables?

The current family of TFMs is limited by context size, however, still reaching up to 500K rows (Qu et al., 2025) on affordable resources. This limit, despite being significantly higher than the earlier TFMs (e.g., TabPFN v1), can prevent learning rules in very large tables.