# Shopify's Continual Learning Loop with PyTorch and vLLM How Shopify achieved 96% cost reduction while surpassing frontier model quality - **Source:** [PyTorch Blog](https://pytorch.org/blog) - **Published:** 2026-09-22 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/db9a0170-61a1-403f-bb02-4b6e3abddf0d ## Frontier model limitations Shopify identified three key limitations of frontier models in production: they're expensive to serve at scale (costing millions annually), don't learn from production failures automatically, and remain general-purpose rather than product-specific. The frozen weights mean production knowledge accumulates in discrete artifacts like prompts and code rather than improving the model itself. This creates inefficiencies as usage scales, particularly for Shopify's GraphQL agent handling 2,000 requests per minute. ## Quality as reward signal The team established a rigorous quality rubric with scored criteria (completeness, execution, response quality, safety) that serves as the foundation for learning. Two expert annotators validate the rubric's clarity by scoring 25 random samples, measuring inter-annotator agreement using Cohen's kappa. This rubric becomes the judge's first prompt and the quality contract for downstream processes. Crucially, it includes randomly sampled production traffic rather than just curated examples, capturing real-world performance nuances. ## Judge calibration process Shopify uses DSPy with reflection-based optimizers like GEPA and Agentic Context Engineering to calibrate the judge. GEPA evolves prompts by analyzing failure traces while maintaining a Pareto frontier of candidates. ACE builds structured playbooks through incremental edits. The team validates the judge by backtesting against previous A/B tests and running targeted degradation tests to confirm it detects known performance changes in engagement and retention metrics. ## Autoresearch optimization Before touching model weights, Shopify pushes the frontier-powered baseline to its limits through autoresearch - an agent proposes changes to prompts, tool definitions, or harness code; evaluates against the judge; and keeps improvements. This optimization targets the entire production harness rather than individual prompts, as behavior emerges from complex interactions across a large codebase with dynamically assembled prompts and custom control loops. ## Mining hard negatives Once harness improvements plateau, Shopify mines anonymized production traffic for hard negatives - low-scoring conversations that reveal model weaknesses. Real merchant traffic provides diverse challenging cases: partial context, ambiguous requests, business-specific workflows, and varied expressions of intent. These failures automatically enter a self-healing pipeline where frontier reasoning models critique and repair them, creating training signals. ## Self-healing pipeline A panel of frontier models critiques each production failure, with an arbiter merging critiques into a single repair instruction. This hint is injected before the user's turn, and the conversation is replayed. Successful repairs become reinforcement learning trajectories using the judge's score as reward. Unresolved failures go to human annotators via Toloka, who correct conversations using the same rubric that calibrated the judge. ## Two-stage training Training occurs in two phases: first, supervised fine-tuning distills healed trajectories (including reasoning chains) into a smaller model. Then GRPO (a reinforcement learning method) directly optimizes against the quality definition, sampling responses and reinforcing top performers. The pipeline runs daily, adding new trajectories while including previous ones to prevent catastrophic forgetting. PyTorch enables distributed training across GPUs for scalable full-parameter fine-tuning. ## Gist compression To reduce latency and serving costs from long system prompts, Shopify developed gist compression. A teacher model with full prompt and student model with learned gist tokens run simultaneously. A custom PyTorch trainer learns gist token embeddings by matching the teacher's output distribution while keeping weights frozen. This reduces prompt length by 75% (6,000 to 1,500 tokens) with no quality loss, significantly improving performance under load. ## GraphQL agent results The continual learning loop delivered three key improvements for Shopify's GraphQL agent: quality surpassed frontier models through SFT and RL from real merchant needs; serving costs dropped 96% ($27M to $1M estimated annually) using vLLM on PyTorch; and performance improved with 19% faster first-token time and 38% lower end-to-end latency at scale. Gist compression also increased throughput by 16% requests/second. ## Hardware efficiency gains The optimizations created significant hardware efficiencies: identical GPUs handle 12% more output tokens per second, requiring 14% fewer GPUs for the same traffic. The combination of gist compression and continual learning makes the feature economically sustainable at Shopify's scale, transforming it from a costly experiment to a production-ready solution available to all merchants without infrastructure strain. ## Compounding learning advantage The durable advantage isn't just the current model improvements but the self-reinforcing loop that continually translates production experience into better weights. Each cycle begins with a more capable model rather than just better prompts or harness code. This creates compounding improvements where a smaller, specialized model becomes faster, cheaper, and more effective than frontier baselines for Shopify's specific use cases. ## Sources covering this story - https://pytorch.org/blog/how-shopify-built-a-continual-learning-loop-with-pytorch-and-vllm/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._