Ollaya: Open-Source Decision Models with Jev-Style Calibration
Run private, calibrated decision models on your own hardware in milliseconds.
1Instant decision-making
Ollaya provides calibrated answers in milliseconds by running decision models in a single forward pass, eliminating token-by-token generation delays. On an RTX 4090, a five-question request takes just 10ms end-to-end through the HTTP API. The system supports TypeSafe's request and response shapes, allowing seamless integration with existing workflows. Local execution ensures data never leaves your hardware.
2Model variety and specialization
Ollaya offers multiple models tailored for different needs: 'laya' for speed, 'decider' for accuracy, 'von' for long context (8,192 tokens), and 'qwen3guard' for content safety. Each model's performance metrics (accuracy, speed) are transparently displayed. The system plans to expand with GGUF LLM-based models via llama.cpp, broadening its decision-making capabilities while maintaining open-source accessibility.
3Local execution advantages
The platform runs entirely on your machine using ONNX Runtime, supporting both CPU and NVIDIA GPU acceleration. Default localhost (127.0.0.1) operation ensures sensitive data like tickets and emails never leaves your environment. GPU acceleration brings response times down to milliseconds, with no per-token fees or API metering - you're limited only by your hardware's capacity.
4Open-source integrity
Ollaya uses weights directly from authors' Hugging Face repositories, pinned to specific commits with sha256 verification. The runtime carries an Apache-2.0 license, guaranteeing transparency and modification rights. Unlike proprietary solutions, Ollaya never re-hosts weights, maintaining the original distribution chain while adding its calibration layer.
5Cross-platform compatibility
Available as desktop apps (macOS, Windows, Linux), command-line tools, and Docker images, Ollaya supports diverse deployment scenarios. NVIDIA GPUs require driver R580+; installers automatically fetch CUDA libraries when detected. Apple, AMD, and Intel GPUs fall back to CPU execution. The system works in WSL 2 and standard Linux environments, offering flexibility across development setups.
6Calibration and thresholds
Models ship with pre-calibrated probabilities that users can threshold based on their risk tolerance. A Modelfile feature allows refitting this calibration using your labeled data. This enables teams to balance decision confidence with operational needs - for example, blocking high-risk git commands (like force pushes) while allowing routine operations with lower confidence thresholds.
7TypeSafe integration
Ollaya maintains compatibility with TypeSafe's Python SDK (v0.7.1) and endpoint shapes, requiring just environment variable adjustments to switch from hosted to local services. The /v1/systemone and /v1/models endpoints mirror TypeSafe's API, allowing existing implementations to work unchanged. Direct curl commands demonstrate the simple JSON request/response format for custom integrations.
8Privacy-focused architecture
By processing sensitive data (invoices, support tickets, user messages) where it already resides, Ollaya eliminates cloud transmission risks. The system's design philosophy prioritizes data sovereignty - weights come from trusted sources, execution stays local, and the open-source license ensures auditability. This makes it particularly valuable for industries handling regulated or confidential information.
Original article: Hacker News — Ollaya: Open-Source Decision Models with Jev-Style Calibration