Pinterest and Intel Scale Open-Source Profiling for Infrastructure
Pinterest collaborates with Intel to enhance gProfiler for large-scale production performance visibility.
1Unified Performance Visibility
Pinterest partnered with Intel to extend gProfiler, an open-source tool designed for continuous performance monitoring. The project aims to provide visibility across a massive infrastructure fleet supporting 640 million monthly active users. By consolidating data from multiple languages like Java, Python, and C++, it eliminates the need for fragmented, manual profiling tools.
2Orchestration Over Proprietary Silos
Instead of building a proprietary platform, the team chose to extend an existing open-source project. gProfiler acts as an orchestration layer, managing tools like perf, async-profiler, and eBPF-based profilers. This approach allows engineers to merge stack traces from different runtimes into a single, unified flame graph without requiring any application code changes.
3Hardware-Level Telemetry Integration
The collaboration integrates Intel PerfSpect to capture critical hardware metrics, including cache behavior, memory bandwidth, and CPU stalls. By correlating software functions with hardware performance events like branch mispredictions, engineers gain deeper insights into how code interacts with physical processors. This adds a vital layer of hardware-level observability to standard runtime profiling.
4Wall-Time Profiling Capabilities
Beyond standard CPU profiling, the updated tool introduces wall-time profiling to identify performance bottlenecks caused by waiting periods. Services often experience latency due to I/O operations, locks, or remote procedure calls, even when CPU usage remains low. This feature makes these hidden delays visible, allowing for more accurate diagnosis of slow application performance.
5Operational Efficiency at Scale
The system is designed for massive scale, with agents consuming less than 1% CPU and 100 MB of memory. Profiling can be triggered remotely via UI or API, removing the need for engineers to gain shell access to individual machines. This transforms profiling from a reactive task into a standard, automated operational capability.
6Robust Data Architecture
The architecture separates collection from analysis, utilizing S3 for storage and SQS for asynchronous indexing. Data is stored in ClickHouse to facilitate complex analytical queries across hosts and services. A React-based frontend enables engineers to explore flame graphs and compare performance profiles across different time periods and infrastructure clusters effectively.
7Strategic Tooling Choices
Pinterest opted for gProfiler over Intel VTune, noting that VTune is primarily a performance-engineering tool rather than a cloud-native fleet profiler. While tools like Parca and Grafana Pyroscope also exist in the continuous profiling landscape, gProfiler distinguishes itself through its specific emphasis on multi-runtime orchestration and deep integration with Intel hardware telemetry.
8Shifting to Continuous Observability
At Pinterest's scale, manual reproduction of performance issues is prohibitively expensive and time-consuming. Continuous profiling establishes a performance baseline, while dynamic capabilities allow for immediate investigation of anomalies. By integrating these insights, the company is effectively treating profiling as a core layer of production observability, ensuring long-term infrastructure efficiency and stability.
Original article: InfoQ — Pinterest and Intel Scale Open-Source Profiling for Infrastructure