Fifty Days to Score Your Customers, or Half a Second
A new class of AI model promises to replace the model-building step entirely. I tested it on four real business datasets. It is as accurate as what you already have, and millions of times more expensive to run.
Key Takeaways
The pitch is “no project”, not “more accuracy.” Tabular foundation models skip feature engineering, tuning, and retraining entirely.
On accuracy, it is a tie. Give your existing model the full customer file instead of a sample and the gap disappears. Four datasets, four ties.
The cost is not comparable. Scoring a million customers takes half a second with the tooling you already run, about fifty days with the foundation model.
The best one is not licensed for commercial use. Google’s weights, and its main rival’s standard download, are research-only. You can install either in one command and be in breach.
My take: a prototyping accelerator, not a production system. Revisit in a year.
What is being sold
For twenty years a prediction has meant a project: collect the data, engineer features, train, tune, retrain forever. Most of the cost and fragility live there, not in the prediction. Tabular foundation models delete it. Hand over the spreadsheet and it answers, no training step. If it holds, it changes how every predictive project is staffed. So I tested it on four real datasets: credit, lending, marketing, telecom churn.
What I found
On accuracy, a draw. The famous benchmarks pit it against a tree trained on a small sample, because it cannot afford a large one. Fair science, poor business case: in production nobody withholds data from a model that can use it. Give the tree the full file and it closed the gap on all four datasets, matching the foundation model to the decimal on bank marketing.
On cost, not close.
| Scoring one million customers | Time |
|---|---|
| The tooling you already run | about half a second |
| Tabular foundation model | about fifty days |
On the messier telecom data it needed over a hundred days for a million, seven hours for three thousand. The reason: a tree learns once and predicts cheaply forever, while a foundation model re-reads your whole file every time. Your data is the running cost. Faster hardware shrinks the gap a thousandfold and it is still tens of thousands of times slower.
Time, cost, quality
Most model comparisons score only quality, which is exactly how a pilot becomes a production problem.
Quality is a tie, so it cannot be your reason to switch. Time is where it wins: nothing to tune, retrain, or maintain, so for a proof of concept due Friday the case is real. Cost is the catch, and it lands on day one of production and never stops.
The fine print
The licence. The best model in the study is the one you are least able to use, and no leaderboard has a column for it. Get legal to look before your data scientists benchmark.
The category. Three tools sell under one label, and on the telecom data they were further apart than the whole traditional field. The leading one has no technical report, and a version from weeks earlier gave silently wrong answers. Buy a specific tool, not the category.
Where it fits
Reach for it on prototypes and one-off analyses, on the permissively licensed options: the saving is real, the serving cost never arrives. Keep it out of any pipeline serving at volume, where the arithmetic settles the matter before accuracy enters. And before shopping for a new model, check the one you run is not starved of data: on three of four datasets, the full file beat switching to anything new, at no cost.
Every number here is a single run. Full method and code in the technical companion.
Code, datasets and every number behind this piece: tabular-fm-scorecard.