# Meta Uses Idle AI Inference Capacity to Enhance Ad Recommendations New architecture leverages volatile opportunistic compute across fleets without degrading baseline latency. - **Source:** [Facebook Engineering](https://engineering.fb.com) - **Published:** 2026-10-12 - **Topic:** [Software & Engineering](https://brifing.si/c/engineering) - **Original article:** - **On Brifing:** https://brifing.si/s/8df1ffe8-a447-406e-bcb5-90b73fdfd187 ## The Inference Paradox Modern machine learning fleets often suffer from a capacity paradox. Infrastructure remains globally constrained due to strict service-level agreements, yet substantial compute pockets sit locally idle. These pockets stem from peak-load provisioning buffers, regional resource-ratio mismatches, and hardware heterogeneity across different accelerator generations. ## Exploiting Volatile Compute Meta aggregated fragmented idle hardware into a shared opportunistic capacity pool. Because this opportunistic compute is preemptible and lacks long-term guarantees, workloads must exhibit extreme elasticity, sustain output quality gains across varying scales, and maintain minimal cross-service dependencies to avoid destabilizing core production systems. ## Dual Elasticity Levers To safely monetize fluctuating compute, Meta implemented two complementary elasticity levers: Rank More Ads and Serve Better Models. These mechanisms dynamically expand or contract non-essential ranking tasks according to available opportunistic resources, maximizing ad relevance without requiring infrastructure expansion or new hardware orders. ## Ranking More Candidates The Rank More Ads lever dynamically increases candidate pool depth across ranking stages during idle capacity windows. Evaluating larger candidate sets boosts recommendation quality but demands balanced hardware expansion. Because retrieval and feature extraction require CPU processing alongside accelerator compute, this lever requires balanced resource surpluses. ## Serving Complex Models Serve Better Models shifts request volume toward computationally heavy, high-accuracy machine learning models as surplus hardware appears. Because input request counts remain constant, upstream CPU demand stays flat. This architectural property allows Meta to exploit accelerator-only surpluses that candidate-expansion mechanisms cannot utilize effectively. ## Strategic Lever Pairing Neither elasticity lever alone can absorb all idle fleet capacity. Combining candidate volume adjustments with dynamic model routing allows the platform to capture diverse hardware imbalances. Meta periodically recalculates compute allocation between both levers based on real-time empirical performance and incoming resource compositions. ## Physical and Logical Isolation Protecting baseline serving traffic requires strict compute boundaries. Complex models use physical isolation on separate routing targets, preventing sudden preemption from affecting baseline replicas. Conversely, candidate expansion relies on logical isolation, enforcing global budget caps calculated directly from active opportunistic instances across fleets. ## PID Feedback Loops Candidate volume is regulated via closed-loop feedback driven by central proportional-integral-derivative controllers. The system continuously monitors downstream tail latencies across serving clusters. When observed latency drops below safe thresholds, the controller scales candidate allowances upward until reaching optimal sustainable throughput limits. ## Value-Aware Request Routing Dynamic model routing relies on an explore-and-exploit framework segmenting traffic by predicted engagement likelihood. The controller measures incremental value gains per segment rather than absolute request worth. Periodic calibration loops keep actual request distributions aligned with target coverage across varying geographic traffic conditions. ## Multi-Layer Load Shedding Sudden resource preemption can outpace central feedback loops operating on minute-level intervals. To prevent cluster saturation, Meta deploys millisecond-level server-side and client-side load shedding. When hardware is reclaimed unexpectedly, servers and clients instantly drop low-value opportunistic evaluation passes before systemic latency degrades. ## Expanding Operational Horizons Meta is actively extending system capabilities through precomputation elasticity, shifting eligible ranking workloads ahead of live user requests. Engineering teams are also improving model portability across accelerator architectures and expanding cross-region inference pathways to aggregate fragmented compute pools across disparate geographic data centers. ## Architectural Significance Turning opportunistic inference capacity into reliable recommendation quality transforms fixed operational overhead into tangible value. Meta demonstrates that combining decoupled elasticity levers, PID-driven feedback loops, and multi-tier load shedding enables large-scale ML systems to safely harvest volatile hardware without violating production reliability constraints. ## Sources covering this story - https://engineering.fb.com/2026/10/12/ml-applications/turning-opportunistic-ai-inference-capacity-into-better-ads-recommendations/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._