How Pinterest Built Its VLM Serving Stack on NVIDIA Dynamo
Scaling vision-language models for next-gen visual discovery experiences
1The Need for VLM at Pinterest
As a visual discovery platform, Pinterest requires AI systems that understand both images and text. Vision-language models (VLMs) power next-gen features like Pinterest Assistant, hybrid search, and content safety. These models interpret images, compare visual candidates, and respond to user intent naturally. Pinterest's strategy involves customizing open-source models to handle product-scale demands, with VLMs becoming central to their AI roadmap.
2Pinterest Assistant Challenges
The Pinterest Assistant exemplifies VLM complexity - it's a multi-turn conversational interface processing both text and images. Serving it requires low-latency inference with proprietary multimodal embeddings. Engineers reworked Qwen3-VL to reduce runtime costs while improving performance. Each request may carry multiple images, requiring complex preprocessing and creating variable computational loads that differ significantly from text-only LLMs.
3VLM vs Text-Only Serving Differences
VLM serving introduces unique challenges compared to traditional LLMs: requests carry image payloads requiring download/preprocessing, vision encoder computation adds overhead, prefill costs become larger and more variable, and KV cache pressure increases dramatically. Pinterest's use cases sometimes require processing thousands of images per request, necessitating specialized infrastructure to maintain performance.
4NVIDIA Blackwell GPU Advantages
Pinterest's solution leverages NVIDIA Blackwell GPUs, which incorporate architectural innovations specifically for demanding AI workloads. Key benefits include higher BF16/FP8 compute throughput (enabling faster processing), increased memory bandwidth (handling large visual data), and larger HBM memory capacity (managing extensive KV caches). These features collectively enable dramatically higher inference performance for VLMs.
5Dynamo's Distributed Orchestration
NVIDIA Dynamo provides the distributed inference layer that powers Pinterest's VLM stack. It enables optimization across the full serving path through several key capabilities: disaggregated encoder/prefill/decode serving, multimodal support in Dynamo frontend and vLLM, KV-aware routing with custom payloads, and KV cache offloading. This flexibility was crucial for handling Pinterest's diverse multimodal workloads.
6Request Payload Complexity
VLM serving requires handling complex payloads containing both text and images (as URLs, base64, or embeddings). The system must download/load images, run preprocessing (resizing, normalization), map them to the model's input format, and build mixed prompts. With Pinterest use cases sometimes involving thousands of images per request, this creates significant overhead compared to text-only serving's simple tokenization.
7Prefill Dominance in VLMs
Unlike text models where decoding dominates, VLM serving sees prefill become the bottleneck due to costly visual context encoding. This creates larger, more irregular KV caches requiring specialized handling. Pinterest's solution implements KV-aware routing and cache offloading while carefully designing prompts to stay within latency and memory service-level objectives (SLOs).
8Multi-Turn Workload Challenges
Multi-turn VLM conversations combine long dialog history with evolving visual context (like revisiting images across turns). This makes prefill heavier and complicates visual state representation. Pinterest had to develop benchmarks reflecting realistic multimodal interaction patterns rather than simple single-shot prompts used in text model evaluation.
9KV Cache Memory Management
VLM serving faces extreme KV cache pressure from large visual contexts and long conversations. Pinterest treats KV as a first-class constraint, using specialized techniques like KV-aware routing, offloading, and disaggregated Encoder/Prefill/Decode designs. This prevents frequent evictions and maintains throughput under prefill-heavy multimodal traffic.
10Custom Model Support Requirements
Unlike interchangeable text models behind standard APIs, VLMs require model-specific support for image fields, multimodal content arrays, and custom projections. Pinterest's serving stack explicitly understands these payload shapes and capabilities. Deployment artifacts and routers encode and route these richer, non-uniform multimodal requests correctly.
11Architectural Innovations
Pinterest's solution (shown in Figure 1) builds on Dynamo's multimodal capabilities while addressing VLM-specific challenges. The architecture handles image preprocessing, complex payload assembly, KV cache management, and multi-turn state maintenance. It represents a significant advancement over traditional LLM serving infrastructures.
12Industry Impact
Pinterest's VLM serving stack sets a benchmark for visual platforms adopting multimodal AI. By solving the unique challenges of vision-language model serving at scale, they enable next-generation experiences like visual conversational assistants and hybrid search. The NVIDIA Dynamo implementation demonstrates how specialized infrastructure unlocks VLM potential while meeting production requirements.
Original article: Pinterest Engineering — How Pinterest Built Its VLM Serving Stack on NVIDIA Dynamo