The Future of Data Systems as AI Costs Plummet
How will data systems evolve in the era of virtually free intelligence?
1The Dramatic Drop in AI Costs
The cost of AI technologies is plummeting rapidly. While GPT-4 class capabilities cost around $30 per million tokens in early 2023, today that figure has dropped below $1. Some providers are pushing costs below $0.10. This decline is making AI nearly free for everyday knowledge work.
2New Challenges and Opportunities for Data Systems
The near-zero cost of AI brings three fundamental challenges and opportunities for data systems: data systems for agents, data systems managed by agents, and data systems created by agents. This new era requires a redesign of data systems.
3Data Systems for Agents
Agents are becoming the dominant workload for data systems. Thousands of agents are spun up in response to each user request. Given the different characteristics of agents compared to humans or applications, data systems need to be redesigned for these agentic users.
4Data Systems Managed by Agents
As agents take on the bulk of knowledge work, a new infrastructure is needed for thousands of agents to manage state over long-running tasks, coordinate, and handle failures. What should such data systems look like?
5Data Systems Created by Agents
Agents are rapidly becoming capable of synthesizing entire data systems in one go. This means custom systems can be rebuilt for each new workload. However, verifying that these systems match intended behavior is a challenge.
6Data Systems Redesigned for Agents
The way agents query databases differs from humans or BI tools. Agents perform a high-volume, heterogeneous workflow including schema introspection, column exploration, and query formulation. How should data systems be optimized for such agents?
7Optimization Opportunities for Agents
Agent requests offer various optimization opportunities. For example, in a text-to-SQL benchmark with multiple agents attempting each task, only 10-20% of sub-plans are unique. This means 80-90% of sub-queries perform duplicate work. How can data systems exploit these properties?
8Proactive Data Systems for Agents
Instead of thinking of data systems as passive executors of queries, they can be made proactive. Data systems have more grounding in data and system characteristics that agents may lack a priori, and can steer agents in different directions.
9High-Level Primitives for Agents
Rather than agents sending one SQL query at a time, they could send a batch of queries. This requires data systems to support higher-level primitives instead of requiring agents to enumerate each SQL query explicitly.
10DBT-Style Macros for Agents
DBT-style Jinja macros can be used for agents to interact with data systems. These macros provide loop-based primitives, enabling agents to work more effectively with data systems.
11Materialized and Virtual Views for Agents
Data systems can prepare materialized and virtual views for agents in advance. This may be cheaper and more effective than having agents author or use them.
12The Intertwined Future of Data Systems and Agents
The future of data systems and agents is intertwined. As these three challenges intersect, the relationship between data systems and agents will become even more complex. This new era requires a redesign of data systems.