Meta's Two Architectural Breakthroughs in Ads Ranking
Sequence learning and LLM-style scaling laws boost ad performance.
1Meta's Ads Ranking Challenges
Meta's ad recommendation systems must process millions of candidates per second to rank thousands of ads. At this scale, some sequence models use hybrid configurations with separate models for user event sequences and sparse feature interactions. However, this approach creates issues like knowledge transfer loss and scaling limits.
2The Power of Sequence Learning
In a 2024 post, Meta demonstrated that modeling the order and timing of user actions produces richer representations of user interests and ad preferences compared to static features. This approach leads to more effective ad recommendations.
3Multi-Stage Sequence Model
Meta introduced two architectural innovations to scale sequence learning: a multi-stage model decoupling offline user modeling from online ranking tasks and a learning technique based on dense tokenization and target-aware attention. These innovations resolve the core tension between model complexity and serving efficiency.
4Offline User Model
The offline user model processes long user histories asynchronously and generates precomputed embeddings capturing deep behavioral patterns. This model strictly separates user features from ad and context features, ensuring user embeddings remain independent of any ad candidate.
5Online Ranking Model
The online ranking model combines representations from the offline model with real-time ad candidate information. This stage is optimized for speed, meeting strict latency budgets while leveraging deep representations computed offline.
6Dense Tokenization
The dense tokenization approach integrates sparse features with sequential behavioral data into a single dense vocabulary. This allows attention mechanisms to discover interactions independently, differing from traditional recommendation systems.
7Target-Aware Attention
Tokenized sparse features and ad candidate information are fused with user behavior sequences and processed by a memory-efficient multi-head attention mechanism. This enables each layer to weigh a user's past behaviors against the specific ad being scored.
8Predictable Scaling Laws
The multi-stage sequence model demonstrates the emergence of predictable scaling laws when running on real-world ad traffic. This balances model performance with computational resources, improving both the ad experience and campaign performance.
Original article: Engineering at Meta — Meta'nın Reklam Sıralama Modelinde Çığır Açan İki Mimari