# Amazon Cuts AI Costs by Replacing LLMs with Decision Models Switching from Claude Sonnet to Strands Decider saves Amazon millions in annual token fees. - **Source:** [Hacker News (news.ycombinator.com)](https://brifing.si/source/13c923cd-6d99-442b-9b8b-1181f4d8cd91) - **Published:** 2026-11-10 - **Topic:** [technology](https://brifing.si/c/technology) - **Original article:** - **On Brifing:** https://brifing.si/s/2783f59f-e863-45ba-a94c-49971d9ed77f ## Transitioning from Generative AI Amazon engineers Carr and Mehul replaced Claude Sonnet 4.6 with the Strands Decider model for internal ticket labeling. While Sonnet previously balanced intelligence and cost, the new decision model achieves identical performance at a 98% lower price point. This strategic shift aims to save millions of dollars in annual token expenditures. ## Decision Model Mechanism Unlike generative models that produce text, decision models output probabilities for predefined options. By mapping ticket data to a fixed catalog of 20 categories, the system eliminates unnecessary text generation. This approach fits perfectly for classification tasks where the output is a specific label rather than a conversational response. ## Technical Implementation Details The team deployed the E4B Gemma 4 build on SageMaker ml.g5.xlarge instances. Each request processes in approximately 640 milliseconds, utilizing 19 GiB of memory. The infrastructure leverages AWS SQS for event queuing, Lambda for processing, and AppConfig for managing deployment stages, ensuring a scalable and isolated environment for the model. ## Optimizing Token Consumption The cost reduction stems from shorter prompts and the elimination of JSON output requirements. Engineers condensed 20 label descriptions into concise criteria, fitting them within the model's 4,096-token window. By truncating older ticket comments, the system maintains efficiency without sacrificing the context necessary for accurate classification of customer escalations or security findings. ## Performance and Latency Gains Internal data shows a 74% reduction in average Lambda execution duration following the switch. At full utilization, the new setup costs roughly $500 per million tickets, representing a 44-fold improvement over previous expenses. These savings exclude additional efficiency gains realized through reduced Lambda runtime, further lowering the total cost of ownership. ## Critical Lessons in Deployment A surprising discovery during testing was that JSON key order significantly impacted output probabilities, shifting results by up to 0.19. To ensure consistency, the team now sorts keys before every request. This technical nuance highlights the importance of standardized input formatting when serving decision models behind an API for production. ## Risk Mitigation Strategy The team utilized a three-way switch via AWS AppConfig to compare Sonnet and Strands Decider on live traffic. This allowed them to judge disagreements in real-time rather than relying solely on offline evaluations. This safety mechanism meant that reverting to the previous model would have required only a 45-second configuration change. ## Future of Classification Agents This transition demonstrates that decision models are highly effective for closed-choice tasks with high volumes. However, they are not universal replacements; LLMs remain necessary if labels require extensive explanations or if the catalog changes weekly. For classification agents, decision models offer a compelling path toward more efficient, cost-effective AI operations. ## Sources covering this story - https://strandsagents.com/blog/amazon-saves-millions-replacing-sonnet-with-strands-decider/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._