The cost of running AI at enterprise scale is actually falling
One infrastructure report circulating in mid-2026 puts enterprise token prices down roughly 67% year over year, from $18.40 to $6.07 per million tokens, attributing the drop to open-source pricing pressure and the rise of multi-model routing, where a system automatically sends each request to the cheapest model capable of handling it. The same report tracks open-weight models capturing 38% of enterprise token volume in Q1 2026, up from just 11% a year earlier. Whatever the precise numbers turn out to be once independently verified, the direction is unmistakable: the cost of doing meaningful work with AI at scale has been falling fast, and open models are a real part of why.
How small models specifically change the calculus
Open-weight small language models, generally under 14 billion parameters, running models like Gemma, Qwen, Llama, or Mistral Small, are reported to deliver cost reductions of around 75% compared to frontier LLM APIs for production enterprise workloads, with some vendor estimates citing per-token costs an order of magnitude lower than large proprietary models. These numbers vary widely across sources and should be treated as directional, but the underlying logic holds regardless of the exact percentage: for a narrow, well-defined task, a large frontier model is often expensive overkill, and a smaller model fine-tuned for that specific task can match or beat it on quality at a fraction of the cost.
The costs vendors don't put in the headline number
Self-hosting open models isn't free just because the model weights are. One analysis puts the realistic break-even point for self-hosting around 2 million tokens a day of volume, below that, a managed API is simply more convenient and often cheaper once you account for infrastructure. Minimal internal deployments are estimated at $125,000 to $190,000 annually, climbing well past $12 million for enterprise-scale implementations, and that's before counting the in-house MLOps expertise required to actually run and maintain the infrastructure. The permissive licensing (MIT, Apache 2.0) and data control of self-hosting are real advantages, but "free license" and "free to run" are different claims.
What this means for how you should actually build
The right answer almost never is "use one model for everything." It's matching the model to the task: a frontier model for genuinely hard reasoning where quality differences matter, a small open-weight model for high-volume, narrow, well-defined tasks where cost compounds fast, and routing logic that sends each request to the right tier automatically. Getting that architecture right, rather than defaulting to the most powerful (and most expensive) model for every call, is often the single largest lever on AI operating cost available to a growing product, and it's exactly the kind of decision an AI automation engagement is built to get right from the start.