Airbnb Releases Skipper, an Embedded Workflow Engine, as Open Source
Skipper enables durable, multi-step business processes within existing Java and Kotlin service architectures.
1Open Source Workflow Engine
Airbnb has officially open-sourced Skipper, a durable workflow engine previously used internally to manage complex business processes. Available under the Apache 2.0 license, the tool allows developers to define multi-step workflows directly within their Java or Kotlin services. This release includes the core engine, admin UI, and various storage adapters.
2Embedded Architectural Approach
Unlike traditional workflow engines that require a separate, centralized orchestration cluster, Skipper runs directly inside the service process. It leverages existing service dependencies and databases to maintain state. This design choice removes the need for an additional shared infrastructure layer, allowing individual teams to maintain full ownership of their workflows.
3Core Operational Mechanics
Skipper uses two primary abstractions: workflows for orchestration logic and actions for executing side effects. The engine persists progress, enabling recovery from crashes or transient failures by replaying the workflow method. It supports complex requirements like human-in-the-loop waits, signals, and retries, ensuring processes remain consistent even during service restarts.
4Deterministic Execution Requirements
The engine relies on deterministic code to function correctly during replay scenarios. Developers must ensure that clock reads, randomness, and I/O operations occur within actions rather than the workflow body itself. By batching action checkpoints, Skipper optimizes database writes while providing mechanisms to narrow the window for potential data loss.
5Testing and Development Utilities
To facilitate development, the project includes WorkflowTest, an isolated in-memory runtime. This utility allows engineers to simulate time, test signals, and verify retry logic without needing external dependencies. The release also features a Kotlin DSL called Skipper State Machine, designed to simplify the description of complex state transitions and events.
6Target Use Case Profile
Skipper is best suited for teams managing multi-step processes that currently rely on custom retry jobs or complex state tables. It provides a robust alternative for services that require durable execution but want to avoid the operational overhead of managing a separate, external orchestration platform for their critical business logic.
7Future Development Roadmap
The engineering team plans to expand storage support beyond the current MySQL and SQLite adapters to include options like Postgres and DynamoDB. Future efforts will focus on improving tooling for replay debugging and workflow evolution. Community contributions regarding automated compatibility checks and enhanced visualization tools are encouraged to support the project.
8Strategic Operational Trade-offs
Adopting Skipper shifts the responsibility for workflow execution, database capacity, and recovery directly to the service team. While this reduces the number of separately deployed systems, it requires teams to maintain a robust operational environment. It represents a shift toward decentralized, service-owned infrastructure for managing long-running, asynchronous business processes at scale.