The Model You Chose Is Already Behind

It is Friday evening in July, and your best engineer is still at her desk. She's reworking one of your most valuable production applications, since she just found out the foundation model that it runs on will be shut off in October. Well before then she has to find, test, and evaluate other models, and migrate the load. She's figuring this out on her own because this is the first time she's encountered this, and central IT never put up a system to manage this. That is her weekend, and probably her next few.

She only found out yesterday, by chance. On a routine call, one of your hyperscalers' account managers asked whether you were still using the old model, and mentioned in passing that it was being retired. The formal notice had gone out months earlier to a technical inbox nobody monitors. Nobody whose application depended on that model had seen it.

Here is the part that should worry you more than the lost weekend: Models age on a curve. The oldest get retired outright. The next tier still runs but the provider stops giving you room to grow on it. The rest simply fall behind newer models that cost less and do more. She caught this one because it was being retired and someone happened to mention it. You are running many models across many applications. She got lucky on one. Is anybody watching the rest?

Falling behind when nothing's broken

The knock-on effect of this issue at scale goes beyond overworked engineers. For eighteen months the application ran on a model nobody revisited while newer releases undercut it on price and performance, and nothing on any dashboard flagged it, because nothing was broken. The scramble at least announces itself. This just leaves money on the table, and no one is going to flag it for you.

The playbook that used to work

Most executives I work with formed their instincts about platform decisions around data. Data behaves differently from foundation models, and that difference is why continuous re-evaluation sounds like churn rather than discipline.

Think about how you chose a data foundation. You selected a warehouse or a database, you built on it, and you ran it for the better part of a decade. Upgrades were events you scheduled around the business, trading a planned downtime for years of stability and the assumption of better data products. You may have standardized on one. More likely you run several. Either way, each was chosen (or inherited), on the expectation it would still be there in five years, and that expectation was usually right.

AI applications don't work that way, because foundation model release cycles are far shorter than data platform cycles. Generations turn over in months. This hits your business on both ends: you are forced off an old model sooner than you planned for, and better, cheaper options arrive faster than your planning cycle can absorb them. A decision you used to make once has become a decision you have to keep making, and most organizations have no mechanism for a decision that repeats.

By the time the email arrives, you are already late

The providers are not hiding the timetable. OpenAI commits to at least six months of notice for generally available models and as little as two weeks for anything with "preview" in the name to stop responding. Microsoft gives at least sixty days before an Azure model is retired, and Anthropic sixty days for publicly released models. The notice that reached the wrong inbox in the opening is a real failure worth fixing, but fixing it only means you hear the bad news sooner.

The deeper problem is that in most companies an end-of-life notice is the only thing that ever triggers a fresh look at the model an application runs on. There is no standing process to track what the providers have released, evaluate it against the workloads you actually run, promote it where it wins, and decommission what it replaces. A company that has that process treats the October email as administrative: it evaluated the successor in May because the successor was cheaper, and the notice confirms a decision already made. A company without one has no strategy other than the email, and the email jumps the queue ahead of whatever the team had planned.

Capacity behaves the same way, and it is not confined to the cloud providers. Serving capacity is allocated per model, and it follows demand toward the newest ones. Through 2026 the frontier labs have rationed throughput rather than raise prices, tightening limits at peak hours as demand outran the hardware. Standardize on an older model and you may find there is no more of it to be had in the week you need it most.

Not choosing was a choice

Ask who owns keeping your AI current and the answer in most companies is that nobody does, not obviously.

Sometimes central IT treats it as an annual procurement question, if it’s asked at all. More often leadership never set a protocol, so each line of business decides for itself when to move and what to move to, with no standard and no trigger. Some teams stay sharp. Most let their applications age, because there is always something more urgent than upgrading something that already works.

What I see across engagements is that the architecture is rarely what fails. Teams build reasonable systems. What is missing is that nobody has been made accountable for noticing when the ground has shifted, and nobody has been told what should oblige them to act.

You also cannot fix that with a committee. Past a handful of use cases, a central group choosing models can't keep up. The work has to be divided on purpose. The center owns the what and the why: which models are approved, what guardrails apply, and what events trigger a fresh look at a model choice. It also owns the platform everything runs through, the gateway whose metadata ties each model call back to a line of business, which is what makes usage and ownership visible without anyone maintaining a list. The builders own the how: evaluate what is new, adopt it where it makes sense, migrate, decommission the old. Centralize the decision and you get a bottleneck. Leave it to chance and every team does something different, and most do nothing.

What Intuit built, and what to borrow

The clearest example of doing this well is one I watched up close. For several years I worked alongside Intuit as a partner, meeting monthly with their VPs of Data and of AI to work through new model testing and plan migrations together.

What struck me was not just the quality of their engineers, but that they had built a system to enable the whole organization to keep pace without anyone losing a weekend to it. The system is called GenOS, and everything I am describing is public: it's a model-agnostic platform giving developers an extensible catalog of frontier, open-source, and proprietary models- including Intuit's own financial ones, with a runtime layer that selects for the job and an internal leaderboard ranking models by task. Their CTO, Alex Balazs, notes that this model-agnostic approach is exactly how they future-proof the platform.

The leaderboard is the part worth replicating, because what it really did was democratize knowledge. Teams could see which models were winning which tasks, why they won, and the prompts that got them there. Nobody was reinventing the wheel. When Intuit shipped an agent starter kit on that foundation, nine hundred internal developers built hundreds of agents in just five weeks.

You don't need to build GenOS, but you do need what it does, which is make switching models a routine engineering task instead of an improvised one. Pointing an application at a different model takes minutes. What follows is the real work, because a prompt tuned for one model routinely underperforms on another, and while a newer model can now do much of the re-tuning, the research still shows it cannot be trusted to do it unsupervised. A good system buys bounded work rather than no work. What bounds it is a set of real cases with known good answers, versioned alongside the prompts and run against every candidate, so a migration becomes a defined debugging exercise rather than an open-ended experiment.

That expectation also has to reach your partners. If system integrators build your AI, ask them before you sign how they intend to keep the application portable across model generations, and what they will hand over so your own team can run the next migration without them. The answer tells you a great deal about what you're buying.

What two years did to the price of the same result

Stanford's Human-Centered AI Institute tracks the price of capability rather than the price of any particular model. Between November 2022 and October 2024, the cost of running a system at GPT-3.5 level of performance fell from twenty dollars per million tokens to seven cents, a decline of more than 280 times in under two years, measured at fixed benchmark parity rather than by substituting a weaker model.

Put a real workload against that. An application consuming five hundred million tokens a month cost ten thousand dollars a month at the opening price and thirty-five dollars a month at the closing one, for the same task, at the same quality, at the same volume of queries. Nobody had to optimize anything to earn that reduction. The market produced it, but it only reaches the companies that can act on it.

Capability is converging at the same time. Stanford's 2026 index found the leading models from six different labs, Anthropic, OpenAI, Google, xAI, Alibaba and DeepSeek, scoring within a hair of one another in public head-to-head rankings. When every leading model is good enough, the advantage moves to whoever runs the cheapest, most reliable one. Your competitor may not be winning on model quality at all. They may simply be delivering a comparable result more cheaply and reliably, and reinvesting the difference everywhere else.

Ninety-one days is not the problem

Go back to Friday evening, and your engineer at seven o'clock, judging claims one at a time. She is doing by hand what a mature team would have automated: pulling real examples, judging the new model's answers against the old, adjusting prompts, checking again. Whatever she works out stays with her, in her head and a spreadsheet, not in anything the company owns. When the next team hits the same wall in February, they start over.

The weekend and the ninety-one days are the least of it. What your organization actually lost is the exercise itself. It went through all of that and came out the other side with no more capability than it had going in, and it will pay again the next time the frontier moves.

The decision in front of you concerns the system rather than the technology. Whether the next one of these produces a repeatable process or another Friday night for your best engineer is a governance question, it belongs to you rather than to your architects, and you can set the mandate in an afternoon.

Three things to do Monday morning

Fund the test set. Require every AI application in production to have a set of real cases with known good answers, versioned with its prompts and run against any candidate model. This is the asset that makes every future migration routine, and running it costs a typical team a few hundred dollars a month in evaluation API calls.

Assume nobody owns this, and assign it. You won't find an existing owner, because in most companies there isn't one. So name an owner for each AI application, and give them the short list of events that should trigger a fresh look at the model: a new release, a price drop, a capacity limit, an end-of-life notice.

Ask one question in your next business review. What models are we running in production, who owns each one, when was each last evaluated, and what deprecation notices apply. If nobody can answer inside a week, you have found the gap, and the answer costs you nothing to demand.

All thoughts, ideas, and opinions expressed here are my own.

Sources

1. OpenAI, Deprecations, developer documentation, accessed July 2026 (minimum notice periods: at least six months for generally available models, three months for specialized variants, roughly two weeks for preview models; notification by email to accounts actively using the model).
2. OpenAI, batch deprecation notice, 22 April 2026 (more than twenty-five model IDs scheduled for shutdown across 23 July and 23 October 2026, including gpt-3.5-turbo, gpt-4, o1, o3-mini and o4-mini).
3. Microsoft, Azure OpenAI in Microsoft Foundry model deprecations and retirements, accessed July 2026 (not-sooner-than retirement date set at launch, 365 days for generally available models and 90–120 days for preview; at least 60 days' notice before GA retirement).
4. Anthropic, Model deprecations, accessed July 2026 (at least 60 days' notice for publicly released models).
5. Microsoft, Azure OpenAI in Microsoft Foundry Models quotas and limits, accessed July 2026 (quota increase request process; priority given to customers actively using existing allocation; TPM and RPM limits defined per region, per subscription, per model).
6. Scientific American, What is the AI compute crunch, and why are AI tools hitting usage limits?, May 2026 (providers currently prefer rate limiting to price increases under capacity constraint). Contemporary reporting also documents Anthropic tightening peak-hour usage limits in March 2026 and GitHub pausing new Copilot signups in April 2026 on compute grounds.
7. Stanford HAI, The 2025 AI Index Report, April 2025 (inference cost for a system performing at GPT-3.5 level fell more than 280-fold between November 2022 and October 2024, from $20.00 to $0.07 per million tokens). Note: this is a benchmark-parity measure, standardized at an MMLU score equivalent to GPT-3.5, not a like-for-like enterprise invoice. The $10,000 and $35 monthly figures in the text apply those published unit prices to an illustrative 500-million-token workload and are arithmetic, not a measured case.
8. Stanford HAI, The 2026 AI Index Report, 2026 (as of March 2026, leading models from Anthropic, xAI, Google, OpenAI, Alibaba and DeepSeek clustered within 25 Elo points on the Arena leaderboard, shifting competition toward cost and reliability).
9. Constellation Research, Intuit embraces LLM choice for multiple use cases, September 2024 (GenOS AI Workbench includes an LLM leaderboard, prompt management and automated evaluation; CTO Alex Balazs on model-agnostic architecture as future-proofing, Intuit Investor Day).
10. Intuit Engineering, Intuit's Custom LLM Leaderboard: Optimizing Model Selection for Financial Use Cases, October 2024 (leaderboard architecture, evaluation framework, model registry, benchmark management).
11. Intuit, Technology at Intuit, accessed July 2026 (GenStudio provides an extensible catalog of commercial, open source, and proprietary LLMs, including Anthropic Claude via AWS Bedrock, Gemini, LLaMa, and Mistral, alongside Intuit's own financial models).
12. VentureBeat, Inside Intuit's GenOS update, December 2025 (Agent Starter Kit enabled 900 internal developers to build hundreds of AI agents within five weeks).
13. Jahani et al., Prompt adaptation as a dynamic complement in generative AI systems, 2026, as discussed in arXiv:2604.27082, When Your LLM Reaches End-of-Life: A Framework for Confident Model Migration in Production Systems, April 2026 (prompt adaptation is a primary driver of results when switching models; automatic prompt rewriting is not yet reliably effective).
14. Practitioner literature on production LLM migration, 2026 (semantic rather than syntactic failure modes on model swap; versioned prompt repositories tied to validated model versions; canary routing of a small share of live traffic scored against the offline rubric). Note: this is established practitioner consensus rather than a single primary study; a primary source would be preferable but I haven't yet found one.
15. Evaluation API cost for running a modest test set (on the order of a few hundred dollars a month for a team with two to three prompt paths), from 2026 LLM testing tooling surveys. Note: figure is illustrative and order-of-magnitude, drawn from vendor-adjacent tooling estimates rather than an audited benchmark; actual cost varies with test-set size, model pricing, and evaluation frequency.