# Meta's Two Architectural Breakthroughs in Ads Ranking Sequence learning and LLM-style scaling laws boost ad performance. - **Source:** [Engineering at Meta](https://brifing.si/source/28383801-e365-4f15-8f20-49d8c0635946) - **Published:** 2026-08-05 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/84d575f1-f4b7-4fb8-ac98-f0da3e834a58 ## Meta's Ads Ranking Challenges Meta's ad recommendation systems must process millions of candidates per second to rank thousands of ads. At this scale, some sequence models use hybrid configurations with separate models for user event sequences and sparse feature interactions. However, this approach creates issues like knowledge transfer loss and scaling limits. ## The Power of Sequence Learning In a 2024 post, Meta demonstrated that modeling the order and timing of user actions produces richer representations of user interests and ad preferences compared to static features. This approach leads to more effective ad recommendations. ## Multi-Stage Sequence Model Meta introduced two architectural innovations to scale sequence learning: a multi-stage model decoupling offline user modeling from online ranking tasks and a learning technique based on dense tokenization and target-aware attention. These innovations resolve the core tension between model complexity and serving efficiency. ## Offline User Model The offline user model processes long user histories asynchronously and generates precomputed embeddings capturing deep behavioral patterns. This model strictly separates user features from ad and context features, ensuring user embeddings remain independent of any ad candidate. ## Online Ranking Model The online ranking model combines representations from the offline model with real-time ad candidate information. This stage is optimized for speed, meeting strict latency budgets while leveraging deep representations computed offline. ## Dense Tokenization The dense tokenization approach integrates sparse features with sequential behavioral data into a single dense vocabulary. This allows attention mechanisms to discover interactions independently, differing from traditional recommendation systems. ## Target-Aware Attention Tokenized sparse features and ad candidate information are fused with user behavior sequences and processed by a memory-efficient multi-head attention mechanism. This enables each layer to weigh a user's past behaviors against the specific ad being scored. ## Predictable Scaling Laws The multi-stage sequence model demonstrates the emergence of predictable scaling laws when running on real-world ad traffic. This balances model performance with computational resources, improving both the ad experience and campaign performance. ## Sources covering this story - https://engineering.fb.com/2026/08/05/ml-applications/from-user-sequences-to-scaling-laws-a-multi-stage-architecture-for-metas-ads-ranking/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._