Meta Unifies Ad Recommendation Ecosystem With Scaled Generative Foundation Model
GEM centralizes multi-surface ads prediction using custom architecture and low-latency post-training transfer.
1Unified Central Foundation Architecture
Meta transitioned its recommendation systems from isolated per-domain models across Facebook, Instagram, and Reels into a unified foundation model named GEM. This consolidated system applies LLM-style pre-training across multi-surface interaction signals. It optimizes the full advertising funnel from broad awareness to deep conversions within one centralized compute engine.
2Conversion and CTR Uplifts
Deploying GEM generated an 18.9% overall conversion improvement and a 12% ad click-through rate increase across Meta's ad ecosystem. Over the past year, targeted users saw additional conversion uplifts of 5.8% on Facebook, 4.1% on Instagram, and 1.6% on off-site conversion destinations across diverse objectives.
3Bending Traditional RecSys Scaling
Recommender systems historically struggled to exhibit transformer scaling laws due to heterogeneous features and shifting data distributions. Meta engineered the Kunlun architecture, co-designed with latest GPU hardware, enabling 2.5x model complexity growth. This design achieved predictable performance scaling similar to frontier LLMs through deeper networks and longer context.
4Unified Kunlun Layer Mechanics
Kunlun standardizes sequence and non-sequence processing by interleaving lifelong sequence modeling with recommendation-specific cross-interactions. Each layer integrates dynamic attention, a summarization module bridging sequence and non-sequence structures, and mixture-of-experts cross-interaction layers. This layout allows GPUs to process dedicated subsets of expert sub-networks in parallel via expert parallelism.
5Block Attention and GDPA
To lower computational complexity, full self-attention was replaced with block-aligned sparse attention, shifting sequence complexity from quadratic to linear relative to window size. Meta also reformulated per-example feed-forward networks into Generalized Dot-Product Attention, unifying multiple attention-like operations into a single GPU kernel for a 2x forward pass speedup.
6CompSkip and OneSeq Mechanisms
The architecture introduced Component Skip Connection, which selectively deactivates specific sequence modules and routes outputs through residual connections, reducing training FLOPs by 43%. Additionally, OneSeq chronologically unifies heterogeneous event sequences after feature aggregation, executing long-sequence tensor operations on GPUs without operator fragmentation across varying data sources.
7Power Law Sequence Scaling
Meta scaled long user engagement context length by 2x in production, proving infrastructure capabilities up to 25x scaling. Prediction quality demonstrated a consistent power law relative to sequence length. Crucially, larger parameter models extracted disproportionate accuracy gains from longer sequences, establishing a compounding loop between capacity and context.
8Post-Training LoopFM Representation Transfer
Real-time evaluation of giant foundation models inside ad auctions is prohibitively expensive. Meta introduced LoopFM, a structured representation framework that compresses GEM's internal activations across key layers. Downstream models ingest these compact representations alongside deep historical memory, doubling knowledge transfer effectiveness compared to standard distillation without extra inference latency.
9Solaris Speculative Precomputation Engine
To serve updated user-ad representations without blocking serving paths, Meta deployed Solaris for speculative precomputation. Inspired by speculative decoding in LLM serving, Solaris precomputes foundation model embeddings for user-ad pairs likely needed in near-future requests, delivering fresh embeddings just-in-time directly onto the production inference hot path.
10Cross-Surface Multi-Task Generalization
Serving 3.6 billion daily active users across diverse surfaces created fractured prediction models. GEM unifies these by jointly predicting clicks and conversions across all surfaces and objectives. Representations learned on high-volume click tasks directly transfer to data-sparse conversion objectives through cost-free data mixing and a domain expert framework.
11Cluster Parallelism and MFU
Scaling RecSys training across thousands of GPUs required custom low-precision kernels and 5D parallelism strategies co-designed with Meta's cluster infrastructure. These optimizations doubled end-to-end training efficiency to 20-25% model FLOPS utilization while handling 4x total training FLOPs, overcoming traditional GPU memory and communication bottlenecks.
12Strategic Shift in Ad Infrastructure
GEM establishes a unified foundation model standard for large-scale enterprise recommendation systems. By decoupling heavy foundation pre-training from fast downstream inference via speculative representations, Meta demonstrates how LLM architectural principles can be economically deployed to optimize complex, latency-sensitive multi-surface commercial platforms at planetary data scale.