DoorDash Reduces Grocery Reorder Latency Using Precomputed AI Agents
DoorDash optimized grocery reordering by shifting LLM inference to offline batch processing.
1Latency Reduction Strategy
DoorDash successfully reduced grocery reorder latency from 16 seconds down to just 2 seconds. By shifting from real-time LLM reconstruction to precomputing recurring purchase needs, the system avoids expensive computation during live requests. This architectural shift directly resulted in a 34% increase in order rates and doubled shopping list views.
2Offline Batch Processing
The system processes 90 days of consumer purchase history alongside profile signals using batch LLM inference. Deterministic validation checks these results before storing them as reusable shopping list data. This approach ensures that the heavy lifting occurs offline, leaving only lightweight resolution tasks for the actual live user request.
3Catalog Resolution Mechanism
During the serving path, the system resolves items against the current product catalog using a three-tier hierarchy. It attempts to match the original store item, then the product identifier, and finally the search term. This tiered logic accommodates inventory changes and unavailable products without requiring the LLM to repeat historical reasoning.
4Shared Agent Infrastructure
The implementation leverages DoorDash's shared agent platform, which provides orchestration, memory, and model access. Domain teams maintain ownership of agent instructions and evaluation criteria while utilizing reusable infrastructure. This modular design allowed the team to integrate a new Reservations agent in just one week, significantly accelerating their development velocity.
5Design Rationale Shift
Engineer Twisha Jain emphasized that superior agentic systems prioritize deciding which tasks belong in the loop versus those handled ahead of time. By moving recurring purchase analysis offline, the system retains current catalog accuracy while minimizing live request overhead. This design philosophy focuses on efficiency rather than simply increasing model reasoning capabilities.
6Structured Shopping Artifacts
The architecture treats the shopping list as a structured artifact rather than relying solely on conversational history. Consumer edits are treated as authoritative, meaning subsequent agent turns start from the updated list. This mechanism allows for simpler operations like quantity changes or removals to occur without triggering unnecessary LLM round trips.
7Selective Agent Invocation
The system remains intelligent by invoking the Grocery agent only when additional reasoning is required. This includes finding cheaper alternatives based on current prices, promotions, package sizes, and inventory levels. By separating historical analysis from live product resolution, the platform maintains a balance between rapid performance and complex, context-aware decision making.
8Industry Contextual Comparison
While competitors like Instacart utilize AI for meal planning and product discovery, DoorDash’s implementation is distinct in its focus on latency reduction through offline analysis. By decoupling historical purchase patterns from real-time catalog resolution, DoorDash demonstrates a scalable pattern for production-grade agentic systems that require both speed and high-fidelity data.
Original article: InfoQ — DoorDash Reduces Grocery Reorder Latency Using Precomputed AI Agents