Meta Andromeda: Scaling Ads Retrieval Through Model-System Co-Design
Meta achieves billion-ads-per-second throughput by fusing retrieval models and hardware pipelines on NVIDIA GPUs.
1Next-Generation Retrieval Architecture
Meta Andromeda serves as the primary retrieval engine for personalized ads, filtering vast candidate pools into high-quality sets. The 2026 iteration introduces a Mixture-of-Experts architecture, achieving ranking-level performance at retrieval scale. This evolution delivers a 2.4% increase in Facebook conversions since 2025, demonstrating the direct business impact of advanced model-system co-design.
2Breaking Throughput Benchmarks
The system now processes over one billion ads per second, representing a 100x throughput improvement over the 2024 deployment. By scoring every eligible ad with near-ranking quality, Meta maximizes advertiser value within existing latency budgets. This performance leap addresses the massive growth in ad candidate pools driven by generative AI creative tools.
3The Memory Bandwidth Bottleneck
At billion-scale throughput, traditional multi-kernel pipelines suffer from memory bandwidth saturation. Intermediate tensors moving between GPU kernels generate terabytes of traffic per second, creating a significant performance ceiling. Meta identified that shipping data between CPU hosts and remote GPU accelerators was the primary constraint, necessitating a fundamental architectural shift in system design.
4Fused GPU Kernel Innovation
Meta developed a fused GPU kernel that executes the entire per-user-ad computation in a single pass. By keeping intermediate results in registers and shared memory rather than materializing them to High Bandwidth Memory, the system eliminates redundant traffic. This approach achieves over 10 billion ads per second on a single GH200 GPU.
5Co-Location Strategy Implementation
To minimize network latency, Meta co-located index matching, scoring, and top-K selection on the same GPU server. Instead of shipping large candidate lists across hosts, the system sends compact request payloads. This reduces network traffic by approximately 100x, effectively shifting the system back into a GPU-compute-bound regime for optimal hardware utilization.
6Mixture-of-Experts Integration
The new retrieval model utilizes an MoE gating network that dynamically routes candidates to specialized user experts. This architecture captures diverse, time-evolving interests by integrating high-fidelity signals like LLM-derived semantic features and cross-surface engagement events. It delivers near-ranking-level quality while requiring 300x fewer inference-time FLOPs than the previous 2024 retrieval model.
7Hardware-Aware Model Design
The architecture is explicitly co-designed with the underlying NVIDIA GH200 execution model. By aligning model complexity with Tensor Core capabilities, Meta ensures that additional expert sub-networks and deeper architectures do not lead to proportional cost increases. This synergy allows for richer feature interactions while maintaining sustainable compute costs at the current massive scale.
8Optimizing Index Matching Logic
Moving index matching onto the GPU required redesigning the index structure for parallel execution. The system now supports complex advertiser-defined targeting logic across tens of feature categories while maintaining real-time update capabilities. This optimization allows index matching to reach throughputs exceeding 100 billion ads per second, significantly outpacing the scoring network's requirements.
9Joint Collaboration with NVIDIA
Early 2026 saw a joint effort to push the retrieval pipeline to hardware limits. NVIDIA provided optimized GPU kernels, while Meta integrated these into the production flow. Each component achieved 2x to 5x speedups, with the gating network reaching 85% of peak HBM bandwidth, validating the effectiveness of the co-design approach.
10Quantization and Efficiency Gains
The team implemented INT8 quantization to further boost throughput by up to 2x over optimized FP16 kernels. Crucially, this was achieved without sacrificing model accuracy. By maintaining 16-bit precision performance at 8-bit computational costs, Meta successfully expanded the headroom for future model growth within the same hardware footprint and power envelope.
11Future Platform Scaling Plans
Building on Hopper-based results, Meta is extending co-design work to the Grace Blackwell platform. The focus is on optimizing CPU-to-GPU compute ratios to suit Andromeda-like workloads. By evaluating sensitivity to these ratios, the team aims to further refine end-to-end performance as they transition to next-generation hardware architectures for personalized ad retrieval.
12Strategic Importance of Co-Design
The success of Meta Andromeda proves that model quality and system architecture must evolve in lockstep. By eliminating bottlenecks through fused kernels and co-location, Meta has established a new efficiency frontier. This methodology ensures that as ad candidate pools continue to grow, the system remains performant, scalable, and economically viable for advertisers.