When somebody says "we should add AI to this", what they almost always mean is "bolt a large language model onto it". A chat box. A summarise button. Something that talks back. I have been guilty of this assumption myself, and I should know better.
So here is a confession. I spent a couple of weeks rebuilding an old prediction model with AI as my collaborator, fully expecting the headline to be "look at the clever new model". It was not. The single most valuable thing the AI did over those two weeks was tell me that the accuracy numbers I had been quietly proud of were meaningless. Not wrong. Meaningless. There is a difference, and I am still slightly embarrassed about it.
The thing I handed over
A while back I had built a classifier to predict an outcome I cared about, then never deployed it. Classic. It was a perfectly respectable bit of classical machine learning: an ML.NET FastTree classifier, gradient-boosted trees, with somewhere around eighty features. No LLM anywhere near it, and rightly so. Recently I dusted it off and handed it to the AI to rebuild.
I gave the AI two things. The model, and a brutally honest list of everything I thought was wrong with it. I half expected it to take the list, retrain the trees, hand me back a slightly shinier version, and we would all go home. Instead it refused to start with the model at all. It started with my data and my label, which in hindsight is exactly where it should have started, and exactly where I had not.
Three questions before you pick an algorithm
Rather than retrain anything, the AI reframed the entire problem around three questions to answer before choosing any algorithm. I have stolen these permanently now.
First: is the label clean? Are you actually measuring the thing you think you are measuring? Second: is there a real decision waiting on the other end of the score, or are you generating a number nobody will ever act on? And third: is the underlying data a moat or a commodity? If anyone could reproduce your dataset from public sources, the model is the easy part and you are competing on something everyone has.
None of those three questions mention an algorithm. That was the point. You can be brilliant at the modelling and still be solving a problem that does not exist because your label is lying to you.
The label was lying to me
And it was. My original label was lazy: any record that ended within the next ninety days counted as the outcome I was trying to predict. Sounds reasonable. It is not.
The AI looked at the actual data and pointed out that "ended within ninety days" is muddied by all sorts of things that are not the outcome at all. A record can end because it was renewed, or replaced, or paused for a while - none of which are the thing I actually cared about. Something that quietly rolls over into its replacement looks, to a naive label, exactly like something that walked out the door for good. They are not the same, and training a model that treats them as the same is how you end up confidently wrong.
What impressed me is that it did not just say "your label is dodgy". It used the data I already had to fix it. There were fields that told a genuine ending apart from a renewal or a replacement, and once those were wired into the definition, a rollover stops looking like a death and starts looking like exactly what it is: a continuation. That is label leakage being quietly removed, using information that had been in front of me the whole time and that I had never thought to use.
What it found when it actually looked
This is the part nobody writes about. The AI went to the real data to ground its design in reality rather than in my assumptions, and two things fell out immediately.
One: the dataset was roughly sixteen times larger than the old documentation claimed. Sixteen times. Every decision I had made about the original model had been made against a mental picture of the data that was years out of date.
Two, and this one genuinely changed the approach: what I had been treating as one population was really several quite different behaviours lumped together under one structure. Different patterns, different lifecycles, different reasons for the outcome, all flattened into a single table and a single model that tried to be all things to everyone. Once you see that, "train one model" stops being the right shape of the problem. You do not get to that insight by tuning hyperparameters. You get to it by letting something look at the real data with fresh eyes and no ego invested in the original design.
When it refused to flatter me
Here is the moment that earned my trust. I looked at an output and asked, more or less, "so is that seven per cent accuracy?"
It told me, politely, that no, that was a base rate, not an accuracy. Only around seven per cent of records hit the outcome in the window, so a model that predicts "it never happens to anyone" is right ninety-three per cent of the time and completely useless. The number I had been eyeing was describing the problem, not the solution. A base rate is not accuracy, and confusing the two is how people fool themselves into thinking a worthless model is a good one. Ouch. Also fair.
It did not stop at the scolding. It insisted on a standing holdout set that the model never trains on, on probability calibration so that a score of "seventy per cent likely" actually means roughly seventy out of a hundred such cases will, and on drift monitoring so the whole thing keeps meaning something as behaviour changes over the months. Training metrics are not real-world performance, and a confidence score you have not calibrated is just a number wearing a percentage sign.
And throughout, it held a firm line that I want to underline given the title of this post: an LLM does not belong as the classifier here. None of this work needed one. The intelligence was in the data, the label and the discipline. Reaching for a language model would have been the expensive, fashionable, wrong answer.
How do you trust a model on day one?
The hardest problem was not technical accuracy. It was trust. How do you trust a brand-new prediction model on the day you ship it, before it has made a single real prediction whose outcome you can check? You cannot point at a track record, because there is no track record yet.
The answer was time-travel backfills, and I love it. The design is an append-only prediction log, where each prediction has its outcome resolved later, once enough time has passed for the result to be real. Then you replay history: you score each record as it was at historical snapshots stretching back twenty-four months, and you check whether the thing you would have predicted actually happened.
There is one piece of discipline that makes or breaks this, and it is sharp. Those historical queries must use dated history, never the current status. The moment you let today's state answer a question about a record as it was eighteen months ago, you have leaked the future into the past, and your backfill will look spectacular and mean nothing. Replaying history honestly means refusing to peek at the present. That is harder than it sounds, because the present is right there and it is so easy to join against.
Done properly, it measures real lift: a holdout group versus a treated group, with Wilson ninety-five per cent confidence intervals around the difference, so you can say with an honest margin whether acting on the score actually changed anything. That is the difference between "the model is accurate" and "the model is worth shipping", and they are not the same sentence.
A small UX decision I keep thinking about
One detail that has nothing to do with statistics and everything to do with not being annoying. A high-risk flag fires once, when something crosses into that risk, and then resolves itself quietly if the signal goes away again. It does not nag every single day. A flag that screams at you daily becomes wallpaper within a week, and wallpaper is exactly what you do not want a risk signal to be. A score is only useful if a human acts on it, and humans stop acting on things that cry wolf. That circles right back to the second of the three questions: is there an actual decision waiting on this number?
What I actually took away
The lesson sits a long way from where I expected it to. "Adding AI" almost never starts with a model, and it certainly does not start with an LLM. It starts with your data and your label. Get those wrong and the cleverest algorithm in the world will faithfully learn your mistake.
And AI as a collaborator is at its most valuable precisely when it refuses to flatter your metrics. A base rate is not accuracy. Training numbers are not real-world performance. A score nobody acts on is a vanity metric with delusions of grandeur. The most useful thing my AI collaborator did across two weeks was not build a better model. It was keep telling me, calmly and with the schema open in front of it, where I had been kidding myself.
If you take one practical thing from this: you can earn trust in a predictive feature before you ship it, by honestly replaying history and checking whether you would have been right. The whole approach lives or dies on one rule, so I will leave you with it. Be ruthless about never letting present-day state contaminate the past. Everything else is detail.
