Ollaya: Open-Source Decision Models with Local GPU Acceleration
Run calibrated decision models privately on your hardware with millisecond latency.
1Local decision model execution
Ollaya enables running decision models locally on your hardware, eliminating the need for cloud APIs. The system processes queries in milliseconds, with a five-question request taking about 10ms end-to-end on an RTX 4090 GPU. It supports both CPU and GPU execution, with NVIDIA GPUs achieving the lowest latency. The platform maintains complete data privacy by keeping sensitive information like tickets and emails on-premises.
2Technical performance benchmarks
Benchmarks show Ollaya's median latency for a five-question request through HTTP API is significantly faster than hosted alternatives. On an NVIDIA RTX 4090, Laya model in fp16 format processes requests in single-digit milliseconds. The system uses a single forward pass architecture without token-by-token generation, contributing to its speed. Calibration metrics show Laya's expected calibration error (ECE) of 0.081 outperforms Jev's 0.246.
3Compatibility and deployment
Ollaya maintains compatibility with TypeSafe's API specifications, allowing existing SDKs to work unchanged against local servers. It offers multiple deployment options including desktop apps for macOS/Windows/Linux, command line tools, and Docker images. The system automatically detects and utilizes available hardware, fetching CUDA libraries only when NVIDIA GPUs are present. All models can run on CPU as fallback.
4Model architecture options
The platform currently supports Laya models from Convai Innovations, including English and multilingual versions plus specialized decision models. Future roadmap includes von models and GGUF LLM-based decision models via llama.cpp. All weights come directly from authors' Hugging Face repositories with SHA-256 verification. The runtime is Apache-2.0 licensed with no per-token fees.
5API and integration features
Ollaya provides REST endpoints compatible with TypeSafe's specifications, including /v1/systemone and /v1/models. The system accepts JSON requests specifying models, states, and questions with configurable criteria. Responses include calibrated probabilities and confidence scores for each decision. Example use cases include customer intent analysis, urgency assessment, and churn risk prediction.
6Security and privacy advantages
By running entirely on local hardware, Ollaya eliminates data exposure risks associated with cloud-based decision models. The system never re-hosts model weights, pulling them directly from original repositories. All processing occurs where sensitive data already resides, whether on employee workstations or company servers. This approach is particularly valuable for financial, healthcare, and customer support applications.
7Cross-platform support
Ollaya provides native packages for all major platforms: .dmg for macOS, .exe/.msi for Windows, and AppImage/.deb/.rpm for Linux. ARM64 Linux and WSL2 environments are supported with CPU-only execution. NVIDIA GPU acceleration requires CUDA 13 and driver R580 or newer. The Docker image supports both amd64 and arm64 architectures with optional CUDA support.
8Developer workflow benefits
The platform significantly reduces development iteration time by eliminating network latency from decision model testing. Developers can prototype with the same models that will run in production, using identical API contracts. Local execution enables rapid experimentation with different model configurations and question sets. The open-source nature allows customization for specific use cases while maintaining compatibility.