Index Insider | Will Local AI Models Change Who Owns the Risk?

Hello. This is Stanton Jones with what’s important in the IT and business services industry this week.

If someone forwarded you this briefing, consider subscribing here.

 

What You Need to Know

Enterprise adoption of local, open-weight AI models is accelerating. Will service providers follow suit?

Data Watch

Local Models Are Moving Fast

Model routing platform provider OpenRouter is reporting that open-weight models now account for about 60% of the token volume routed through its platform. That’s up from about 25% six months ago. When you combine this with the bevy of benchmarks that show that local models generally reach frontier-level capability a few months after the actual frontier-level models, it’s clear that demand for local, open-weight models is growing.

We’re seeing something similar in our research. We just wrapped our annual State of Enterprise AI report, and one of the key areas we dug into this year was around locally hosted, open-weight models.

As you can see from this week’s Data Watch, 65% of respondents indicate they are either evaluating or piloting open-weight models, and nearly 20% have already deployed a local LLM for a production workload.

What’s interesting here is why companies are using local models.

As you can see on the right-hand chart, privacy and security are the reasons selected most often. This is likely related to two questions. First, enterprises are concerned that the frontier model providers will use their data to train future models. Even if the model providers say they are not doing this, it still leads to all kinds of important questions around data privacy and data sovereignty. But there is a second, more strategic question as well: Will frontier model providers use their clients’ proprietary data to learn and replicate the workflows unique to those companies, making the provider a potential competitor in the future?

These are important questions that most enterprises are just starting to work through, and they are clearly driving demand for open-weight models.

What About AI Costs?

We see the rapid rise of local models as part of a hybrid approach to LLMs. We’re likely to see enterprises using a mix of models depending on the workload – similar to what we’ve seen emerge in cloud over the past decade. For example, an accounts payable process does not need frontier-model capability or cost, but refactoring a massive legacy codebase might.

As you move down the Data Watch chart, you see that the next three reasons enterprises are using or considering a local model have to do with cost. Reducing AI operating costs, managing usage at scale and improving the predictability of AI costs are all related to the Wild West of AI economics.

And this is where providers come in. We are seeing a lot of angst in the market today around who owns model risk – the provider or the client. Given the probabilistic nature of LLMs, combined with ever-changing token pricing models, this risk can be significant. For example, what happens when an agent takes a different path to solve a problem, which provides a correct, but unexpected answer, while doing it at two or three times the expected cost?

Providers that start to wrap their services around local, open-weight models can (in theory) start to mitigate some of this model risk for their clients. By adjusting model weights, they can tune the model to amplify their services offering, and at the same time mitigate the cost risk. In this model, inferencing becomes more of a decision for the provider about how much capital to deploy against their AI-enabled offering, rather than building contracting protections around unknowable price changes.

This ownership of the underlying model and weights could have the added benefit of enabling providers to better measure the value clients are getting from their AI-enabled services. And, as we discussed a couple weeks ago, giving clients confidence that value can be measured today will be an increasingly important capability for providers to win tomorrow. Local, open-weight models may be a path to doing just that.

We hope you can join us next week at the ISG Sourcing Industry Conference in Dallas. We’ll be releasing the full State of Enterprise AI Report at the event. Register here.