# Respan: An Open-Source Rust AI Gateway Built on Plugins

A decoupled, modular AI gateway designed for extensibility through a lightweight core architecture.

- **Source:** [Hacker News: Launches](https://news.ycombinator.com)
- **Published:** 2026-10-20
- **Topic:** [Software & Engineering](https://brifing.si/c/engineering)
- **Original article:** <https://news.ycombinator.com/item?id=50167583>
- **On Brifing:** https://brifing.si/s/bd652321-b756-4554-ba92-c03a57fb40ef

## The Shift to Modularity

Respan introduces an open-source AI gateway built in Rust, emphasizing a decoupled architecture where every component functions as a plugin. Unlike monolithic gateways that bundle numerous providers and features, Respan provides a minimal core. This design allows development teams to assemble specific AI infrastructure requirements without modifying the underlying gateway source code.

## Overcoming Monolithic Limitations

The project originated from challenges faced while wrapping LiteLLM, where the codebase ballooned to over 100,000 lines. A critical failure during a January upgrade forced the team to manually backport fixes, highlighting the risks of monolithic dependencies. This experience drove the decision to build a custom, plugin-first architecture inspired by Nginx.

## Core Event Dispatching Mechanism

The gateway operates by handing HTTP or WebSocket connections to client formats that recognize the traffic. The core then executes registered plugins in order, enabling them to filter streams, wrap requests, or make secondary calls. This event-driven model supports both simple HTTP requests and complex, long-lived bidirectional sessions for real-time AI interactions.

## Extensibility Through Plugin Ecosystem

Everything within Respan is a plugin, including provider integrations, protocols like MCP, and security features like PII redaction. Developers can implement custom logic without touching the core, making it simple to add unique providers. For instance, a Mistral provider implementation requires only 22 lines of code, demonstrating the system's lightweight nature.

## Handling Diverse AI Modalities

Respan goes beyond simple model routing by automatically handling tools, images, and reasoning protocols. It allows applications built for the OpenAI Realtime API to interact with Gemini Live through the gateway. By parsing requests into a shared structure while preserving native fields, it ensures compatibility without dropping provider-specific data during transmission.

## Deployment and Open Source

The gateway is available under MIT or Apache-2.0 licenses on GitHub, allowing users to build from source or run it locally. Developers can swap base URLs in existing clients to leverage the gateway immediately. This flexibility ensures that teams maintain control over their infrastructure while benefiting from the modular plugin architecture.

## Business Model and Observability

While the core gateway remains open source, the team monetizes through a hosted platform. This service provides native observability, evaluation tools, and unified billing for organizations. By offloading the operational burden of running the gateway, the platform offers a managed alternative for teams that prefer not to self-host the infrastructure.

## Future Roadmap and Contribution

The primary roadmap focuses on expanding the plugin ecosystem and encouraging community contributions. Documentation in the repository provides guidance for developers to write and share their own plugins. By fostering this collaborative environment, Respan aims to become a standard, extensible foundation for routing, security, and observability in AI agent applications.

## Sources covering this story

- https://news.ycombinator.com/item?id=50167583 — 

---

_Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._