LinkedIn Integrates OpenTelemetry for Full-Stack Member Journey Observability
Connecting client-side interactions to backend services via a unified telemetry platform.
1Bridging Client-Backend Observability Gaps
LinkedIn developed an OpenTelemetry-based platform to unify client-side actions with backend service execution. Previously, engineers struggled to correlate mobile or web interactions with the specific backend requests they triggered. This new architecture allows teams to trace a member's journey from a button tap through to the underlying data store operations.
2Implementing the pageForestId Context
Standard distributed tracing often fails to capture the full scope of a single user experience. LinkedIn introduced the pageForestId to group multiple related requests under one page context. This allows engineers to investigate individual requests while maintaining visibility into the broader activity that contributed to the specific member experience observed.
3Standardizing Across Diverse Tech Stacks
Operating a complex stack involving Java, Python, Go, and Rust requires a common instrumentation model. LinkedIn utilizes OpenTelemetry to ensure consistent trace context propagation across these environments. By building custom wrappers around standard SDKs, the platform manages mobile lifecycle concerns and metadata while keeping instrumentation consistent for various product teams.
4Telemetry Ingestion and Storage Architecture
The system relies on Apache Kafka for transporting telemetry data, which is subsequently stored in ClickHouse. By using pageForestId, traceId, and timestamps as primary keys, engineers can perform efficient correlation. This infrastructure enables a direct drill-down from a member-facing interaction to the specific backend execution responsible for the observed system behavior.
5Coordinated Sampling for Data Consistency
Capturing every span would create unmanageable data volumes, making sampling a critical architectural component. LinkedIn employs coordinated head sampling, utilizing a pseudonymized member identifier to ensure consistent decisions across client and server activity. This prevents scenarios where client-side traces are kept while corresponding backend traces are discarded, maintaining data integrity throughout.
6Managing Client-Side Telemetry Constraints
Mobile applications present unique challenges, as they can be suspended or terminated before telemetry transmission completes. LinkedIn addresses this by batching spans and flushing data when the application moves into the background. These controls ensure that critical observability data is captured despite the volatile nature of mobile client environments and connectivity.
7Privacy and Data Security Controls
Distributed traces can inadvertently expose sensitive member or application information if left unmanaged. LinkedIn enforces strict attribute allowlists, denylists, and redaction mechanisms within its telemetry pipeline. These security controls are essential for maintaining compliance and protecting user data while still providing engineers with the necessary visibility to debug complex production issues.
8Strategic Shift Toward Vendor Independence
By leveraging OpenTelemetry as a foundation, LinkedIn gains greater control over its observability stack compared to relying solely on commercial tools. This approach avoids vendor lock-in for instrumentation while allowing the company to customize its ingestion and storage layers. Ultimately, this connects technical metrics directly to the actual member experience.