Stop Counting AI Tokens, Start Counting What Ships
Shift your AI development metrics from consumption-based token costs to software delivery efficiency.
1The AI Token Trap
Developer AI agents perform iterative planning, coding, and testing, consuming significantly more tokens than simple chat prompts. This high volume creates financial pressure, as seen when Uber reportedly exhausted its 2026 AI budget by April. Relying on per-token pricing for complex agentic workflows often leads to unsustainable costs and restrictive usage rationing.
2Redefining Success Metrics
Blanket spending caps often restrict valuable workflows by measuring consumption rather than output. Organizations should instead track the cost per merged pull request, dividing total AI-assisted development costs by successful code merges. Monitoring rework and defects alongside this metric ensures that lower costs do not inadvertently signal a decline in overall software quality.
3Strategic Model Routing
Not every coding task requires a frontier model. Organizations should reserve premium inference for complex reasoning and novel planning. Routine tasks like code generation, test creation, and refactoring can be routed to open-weight models. A governed gateway manages this distribution based on task complexity, data sensitivity, and latency to optimize overall spending.
4Leveraging Open-Weight Models
Leading open-weight models now approach frontier performance on specific software engineering benchmarks. While they may not be equivalent across every repository, they offer a practical alternative for high-volume tasks. Engineering teams must evaluate these candidates against their own internal codebases, specific tests, and acceptance criteria before assigning them to critical production development workflows.
5Owned Capacity Economics
Rented inference adds costs with every call, whereas owned capacity commits costs upfront, making additional calls cheaper as usage scales. While ownership involves power, maintenance, and depreciation expenses, it provides greater control over proprietary code. Keeping sensitive code on-premises allows teams to utilize AI agents in repositories where security concerns previously stalled adoption.
6Hybrid Infrastructure Results
HPE implemented a hybrid AI platform routing routine workloads from 150 internal applications to on-premises open-weight models. This strategy reduced token costs by 30 to 50 percent. By running high-volume tasks on owned capacity while sending only the most demanding work to frontier APIs, organizations maintain both cost efficiency and access to top-tier models.
7Identifying Power Users
Usage patterns are rarely uniform, as the top 5 percent of users often drive the majority of consumption. Averaging costs across all licenses hides the impact of these power users. Tracking cost per active developer alongside cost per merged pull request provides the granular visibility needed to understand how AI spending scales with headcount.
8Building Sustainable Adoption
Ownership is most effective when sustained demand keeps infrastructure busy, while low usage favors API-based models. Leading companies will not ration AI adoption to control costs. Instead, they will build operating models where increased AI usage improves development economics, ensuring that growth translates into more software shipped rather than just higher variable bills.
Original article: MIT Technology Review — Stop Counting AI Tokens, Start Counting What Ships