Millwright: Self-Hosted LLM Router
Optimize cost and performance with Rust-based solution.
1What is Millwright?
Millwright is a Rust-based, self-hosted LLM router. It supports providers like OpenAI, Anthropic, and Amazon Bedrock. It focuses on cost savings, transparency, and performance. Easy to integrate with CLI interface.
2Features and Benefits
Millwright offers policy-controlled model roles, cheapest healthy route selection, and cache affinity. It provides detailed analysis like per-team costs, cache usage, and model/provider mix.
3Protocol and Support
Millwright supports protocols like OpenAI Chat Completions, Anthropic Messages, and text/tool translation. It offers role-scoped session lanes without serializing concurrent agent traffic.
4Cost Analysis
Millwright provides measured usage and model candidate economics analysis. It offers reports in HTML, Markdown, and JSON formats. Detailed analysis for cost savings.
5Reliability and Setup
Millwright offers reliability features like bounded failover, circuit breakers, timeouts, and concurrency limits. It provides interactive setup without storing provider secrets.
6Why Millwright?
Millwright optimizes cost and performance as a self-hosted solution. It is reliable and fast with its Rust-based structure. Offers easy integration and detailed analysis for developers.
Original article: Hacker News: Show HN — Millwright: Kendi Kendine Barındırılan LLM Yönlendirici