IBM CEO: Rationing AI is a Strategic Failure for Enterprises
Scaling artificial intelligence requires controlling infrastructure costs rather than limiting employee access to tools.
1The Shift to Operational Costs
Global spending on AI has transitioned from initial model training to daily operational expenses. Business leaders now face the challenge of scaling these systems profitably. While many companies currently resort to rationing access to manage budgets, this approach limits productivity and ultimately hinders long-term competitive success in the digital market.
2Engineering Drives AI Costs
AI agents frequently perform repetitive tasks, reading vast amounts of data to generate simple outputs. This redundancy creates significant overhead, similar to early internet bandwidth inefficiencies. By implementing caching mechanisms, companies can ensure systems reuse previously processed data, allowing hardware to focus on new tasks and increasing overall processing capacity.
3Optimization Through Parallel Processing
IBM, in collaboration with Red Hat and Google, developed open-source solutions to optimize AI workloads. By caching data, systems avoid repeating work, enabling hardware to handle seven times more volume. These continuous engineering optimizations are essential for bending the AI cost curve and ensuring that infrastructure remains efficient at scale.
4Strategic Model Selection
Companies often assume that the newest, most expensive models are required for all tasks. However, open-weight models can provide significant value at a fraction of the cost. For instance, one test showed an open model delivered a 52-fold speedup for only 26 dollars, compared to 299 dollars for proprietary alternatives.
5Controlling the Price Structure
Relying on vendor discounts for data processing leaves companies vulnerable to price revisions. True efficiency is achieved by owning the infrastructure where AI runs. By selecting models fit for specific purposes and controlling the underlying hardware, organizations can manage their own cost structures rather than being subject to external pricing.
6Predictability in Probabilistic Systems
AI is inherently probabilistic, meaning the same request can vary wildly in cost depending on the system's path. This unpredictability breaks traditional IT budgeting models based on fixed units like CPU hours. Robust operating layers are necessary to limit tools, ensure reliability, and keep enterprise AI costs within predictable boundaries.
7Data Sovereignty and Security
Approximately 80 percent of enterprise data is unstructured and resides within internal systems like insurance claims or patient records. This data is too sensitive and regulated to be shipped to external vendors. Intelligence must be brought to the data, ensuring security and sovereignty while maintaining control over the associated processing costs.
8Owning the Operating Layer
The AI operating layer connects disparate systems and governs usage, ensuring companies retain control over their economics. If a supplier controls this layer, they effectively control the meter on your spending. By owning this layer, enterprises can scale AI effectively, ensuring they capture the maximum value for every dollar spent.
Original article: Fortune | FORTUNE — IBM CEO: Rationing AI is a Strategic Failure for Enterprises